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ABSTRACT • 

Four author-contributed papers examine issues in 
assessing child progress in early childhood special education. D. 
Bricker and S. Gumerlock present "A Three-Level Strategy" which 
features analysis of daily or weekly progress, analysis of progress 
toward long- and short-term objectives, and analysis of progress 
toward program goals. C. Dunst follows with a discussion of "Four 
Developmental Perspectives" (maturat ional , behavioral, 
process-oriented, and ecological) and suggests several generally 
feasible research strategies. N. Johnson-Martin enumerates three 
"Sources of Difficulty" in assesssing progress in the population: 
the discrepancy between implicit and explicit goals in programs 
serving handicapped infants; (2) the diversity of needs of children 
served in early intervention programs; and (3) the different kinds of 
training of professionals who staff early intervention programs. Irt 
the final paper, p. strain stresses the concept of "Social 
of intervention outcomes, arguing against the use 
tests and advocating instead the use of 
measure progress. (CL) 
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Introduction 



Early childhood special educators face many problems as they try to assess 
child progress. Population variability, diverse handicapping conditions, and 
socioeconomic factors influence the. intervention targets; from birth to age 6 
years, typical rapid development confounds the effects of the program and the 
effects of maturation; infants and very young children have limited behavioral 
repertoires with which to measure change ; instruments appropriate for measuring 
developmental change in severely handicapped or sensory impaired children are 
either noneyistant or notoriously lacking in technical attributes; and most 
early intervention programs do not have resources to apply rigorous research 
designs. 

To explore these problems and some solutions, TADS tapped the ideas of 
several members of the Handicapped Children's Early Education Program. Our 
contributors are: Phillip Strain, Director of the Early Childhood Research 
Institute at the University of Pittsburgh; Diane Rricker^ Director of ths Early 
Intervention Program, University of Oregon, Eugene, assisted by Sarah Oumer- 
lock, doctoral student in early childhood , special education; Carl Dunst, Direc- 
tor of Project SUNRISE, Western Carolina Center, Morganton, North Carolina; and 
Nancy John son -Mar tin „ Investigator with* the Carolina Institute for Research in 
Early Education of the ??andicapped , University of North Carolina at Chapel 
Hill. These experts in the areas of assessment, curricula, and efficacy of 
early intervention responded to three questions : 

What is child progress ? (goals of intervention; appropriate domains 
for measuring progress) " 

How can child progress be measured ? (types of instruments; purposes 
of assessment) 

What designs or procedures should be used to determine if progress 
has been made ? (problems of design; demonstrating progress as a 
result of intervention) * 

The contributors share some points of view within their unique perspec- 
tives on assessing child progress. Of the common themes, the writers agree 
that there is & need to distinguish between assessing child progress for indi- 
viduals (or groups) and assessing program impact. They also share the idea 
that measurement of progress should r*j* et the goals of the program, including 
benefits to the family of which the cbixd is a member. And they share concerns 
about the content of assessment (what to measure), and what kind or amount of 
change constitutes progress. A sampling of the unique perspectives of the 
individual contributors, follows. 



Diane Bricker arid Sarah Gumerlock propose a three-level strategy for' 
assessing child progress. Each level has its own purpose (child change vs. 
program impact), referent (child's present performance compared with past per- 
formance, child's performance compared with' norms, child's performance com- 
pared with program goals), and measurement techniques (trial-by-trail data 
collection for individuals, instruments which reflect long-term goals used 
across children, standardized instruments). They discuss the problems inher- 
ent at each level and suggest some practical solutions. 

Carl Puns t focusses on four major developmental perspectives. Each influ- 
ences the goals of intervention, outcomes to be measured, and techniques of 
measurement. He emphasizes the need to clearly define the intervention as an 
independent variable, pose appropriate questions, and employ feasible research 
strategies to answer the questions. 

Nancy Johnson-Mart in enumerates three sources of difficulty in assess- 
ing progress in young handicapped children. Explicit goals (developmental 
progress) vs. implicit goals (normal life experience) creates conflicts in 
determining what to ; measure. The diverse needs of subgroups of handicapped * 
children dictate that fcte appropriate domains for intervention will vary, as 
will the intervention techniques and measures of progress which follow. 
Further, she says that the professional training of the disciplines involved 
in early intervention influences the way in which problems are ad dres sed and 
progress is measured.. She describes the types of instruments available and 
when and ho;* they are most appropriately used. 

Phillip Strain stresses the concept of social validity of intervention 
outcomes. He argues against the use of standardized tests to measure progress 
of handicapped children. He states that the gains required to produce, for 
example, a ten-point 10 .score increase may be so limited that they are not per- 
ceived by significant others in the child's family or community. He describes 
single-subject designs or, as he prefers to call them, subj ects-as-their-own~ 
controls designs axid believes they are particularly attractive for determining 
child progress in early intervention programs. 



Chapter! 

A Three-Level Strategy 

by Diane Bricker and Sarah Gumerlock 



WHAT IS CHILD PROGRESS? 



At first blush, this seems a straightforward question that should have a 
straightforward answer. However, this is not the case. The question of child 
progress must be related to some benchmark or standard. There must be a 
source of reference, whether it be the child's performance, the performance of 
others, or the test itself (Hamilton & Swan, 1981). Progress cannot be deter- 
mined in the abstract. One must ask: Progress towards what? Measured by 
what? Once a relationship to a target or goal is established, the following 
qualifiers must be considered: How much change? What kind of change? There- 
fore, to answer the question, "What is child progress?" it is first necessary 
to answer these three questions : 

• What content (e.g., benchmarks of development or standards of 
performance) is to be measured? 

• What quantity of change toward acquisition of this content 
constitutes progress? 

• What quality of change towaids acquisition of this content 
constitutes progress? 

Measuring Child Progress: Purpose and Content 

No accepted set of standards exists which permits universal comparison of 
children to determine progress. No operationally defined prototype is 
generally acceptable (Bricker, 1978). For example, all may agree that children 
should do well in school. But, attempts to define "do well in school" high- 
light the need. for. qualifiers . Benchmarks are established or defined in rela- 
tion to the individual's personal environmental and biological resources, per- 
sonal values, and the values of the individual's family and culture. (The 
standards of progress for a biologically normal child are different from those 
for a child with Down's syndrome or for a child raised in poverty.) Child 
progress can be determined in relation to school or program goals, goals for 
specific "populations , or goals for a particular child. Since particular com- 
parisons are appropriate for particular purposes, it is important to determine 
the purpose of measuring child progress before data is collected. 



Child Change and Program Impact 



Child progress Is usually measured In order tp evaluate general program' 
Impact and to monitor child change. Established program goals can provide 
guidelines for selecting the content areas for determining child change and 
program impact. Many early intervention programs lack a cotiesive system or 
plan for determining program goals. This, in turn, affects the selection of 
goals for children participating in the program. Program g;oals should provide 
the broad benchmarks of development or standards of performance against which 
child progress can be measured. 

An organizational framework of three independent, interrelated levels of 
evaluation can help establish the content areas for measuring child progress. 
Figure 1 shows one such framework. The greater the consistency between the 
three levels shown in Figure 1, the more efficient, appropriate, and useful 
will be the selected areas for measuring child change and program impact. 

v 

Figure 1 

Three-Level Framework to Direct Program Evaluation 
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The selection of program goals should he predicated upon the underlying 
philosophy or rationale that sets the general structure and atmosphere for the 
program. A frequent rationale for early intervention is that it helps the 
child grow and develop in important behavioral areas and helps the family 
accept the handicapped child. Based on this rationale , a set of related pro- 
gram goals might include enhancing: a) children's acquisi tion and use of 
sensorimotor skills,. b) children's acquisition and use of social skills, c) 
children's acquisition and use of communication skills, d) children's acquisi- 
tion and use of irtotor skills, e) positive family interaction, and f). parent 
positive attitudes about their children. Such program goals (Figure 1, Level 
3) form a content framework from which professional staff and parents can 
select and develop the individual child's long- and short-term, obj ect ives 
(Level 2). For a specific child,, objectives based on program goals might he: 

• Long-Term Objective: The child will, appropriately use social- 
communicative signals to indicate labels, requests, greetings, and 
to gain attention. 

• Short-Term Objective (based on the long-term objective): The 
child wil look at adults, point to objects, and vocalize to gain 
desired objects, actions, or events. ' „ 

• Long-Term Objective: The child will initiate appropriate inter- * 
actions with peers. • 

Short-Term Objective: The child will respond appropriately to 
interactions initiated towards him by peers. 

The child's individual weekly activity/lesson plan (Level , 1), is based on 
the child-s individual long-and short-term objectives (Level 2). Weekly acti- 
vity plans for the short-term objectives might be: 

• During opening and closing group time, a desired object or event 
will be withheld while the child is prompted to look at adults, 
point , and vocalize. 

• During snack time* "juice and crackers will be placed out of the 
child's reach and the child will be prompted to make eye contact, 
re.ach, and vocalize. 

• During small-group activities, peers will be prompted to initiate 
interaction with the child (e.g., share a crayon, throw a" ball), 
and the child will he prompted to reciprocate (e.g., give peer his 

■ • or her crayon, roll the ball hack). 

• During any program activity the child will be immediately rein-^ 
forced (given desired object, event, person) when he or she spon- 
taneously makes eye contact, reaches, and vocalizes. 

• During any program activity, the child will be immediately rein- 
forced (praised, given another turn) for responding appropriately 
to peer's actions. 

These examples reflect a series of training targets that are global at 
Level 3, then become more specific at Level 2 and Level 1. This system is 
useful for selecting the content to measure child progress at each of the 
three levels of evaluation. The measurement strategy and the comprehensive- 
ness, of the measurement target may change at these three levels, but the basic 
content remains the same. This content consistency provides „a set of guide- 
lines which directs measurement of child change and program impact. 
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HOW CAN PROGRESS BE MEASURED? 



Once the purpose and set of content benchmarks or standards against which 
change will be compared are established, then decisions as to how to measure 
the child against these established, standards must "be made. Consider the pur- 
poses for the evaluation. To evaluate child change, the child's performance 
may be compared with the child's previous performance. However, to evaluate 
program impact, the child's performance may need to be compared to some ' 
external source (such as a standardized test, a set of norms, or a statistical 
analysis of change). The three-level framework in Figure I that provides a 
cohesive strategy for selecting content can also provide a useful systemfor 
measuring child change and program impact. Figure 2 parallels Figure 1 and 
o illustrates appropriate instruments for measuring progress at each of the 
t hree^ level s • 



i Figure 2 - . 

Three-Level Framework to Direct Selection of Instruments for Measuring Progress 
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Level 1: Measuring Progress Towards Daily/Weekly Training Targe ts 

. In most cases, the measurement of the child's acquisition and use of 
specific training targets will have to be conducted without the benefit of a.,, 
standardized or criterion-referenced instrument. Such intruments tend to tar- 
get more general response classes than are or should be reflected in par- 
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ticular training targets for children. To measure child change related to 
particular training targets, a specific data collection format (e.g., trial by 
trial, probes, observational sample) appropriate to the training target will 
need to be devised. Since /particular formats will be used with particular 
targets, measurement approaches probably will vary across training objectives. 
For example, data collected on the' child' s use of vocalizations and gestures 
might be monitored through a weekly observation of the child at play. Moni-~ 
toring the child's responses to his or her mother's vocal imitation might be 
done by conducting daily trial probes. 

The amount and type of data collected should be determined by the 
severity of the child's disability, the specif ic training obj ectiver, and 
program resources. Collection of information on child progress can be done 
with the instructional program. The intervent ionis t can record the child's 
behavior and instruct a new response simultaneously if the data collection 
system has been properly devised to* consider the nature of the instruction, 
the child, and the demands on ,the interventionists. The strategy should 
balance the program resources against program demands — monitoring child 
progress is only one demand. t . 

The Early Intervention Program (EIP) at the Center on Human Development, 
University of Oregon (Bricker, Bailey. & McDonnell, 1984) used "a strategy 
involving a grid format to record data. This system attempts to assess the 
impact of intervention using a group format with the training activities/les- 
son plans integrated into daily, ongoing activities. The data collection grid 
is a flexible format that allows the intervener to target several different 
skills for each child across activity groups (see Figure 3). The grid format 
accommodates individualization, of antecedents, responses, and consequences or 
error corrections; type and frequency of data collection and dispersion of 
instruction; and data collection across set tings , adults , and peer groups. 
The antecedents, responses, atld consequences written on the data grids are a 
shortened version taken from the individual child's program plan (Figure 4). 

Level 2: Measuring Progress Toward Long- and Short-Term Objectives 

The information acquired on child change in terms of specific targets is 
essential to the formulation and implementation, of a sound intervention pro- 
gram. The data collected on child progress toward\long- and short-term objec- 
tives can also be used to measure child change. And, these data can be used 
to evaluate program impact if the nature of the data can be aggregated as 
group data for meaningful analyses • To measure progress toward long- and 
short-term objectives, published instruments with norms or reliability and 
validity information may be useful. Selection of an instrument should be 
based on these factors: - 

• The test reflects the long- and short-term objectives selected for 
the children. - - — ^ 

• The test reflects the program's goals. 

• The test can be administered and scored without much difficulty* 

• The test's results are interpretable and useful. 

For exsmple, the Revised Gesell % Developmental Schedules (Knobloch, 
Stevens, & Malone, 19R0) may be appropriate for- measuring achievement for a 
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Figure 3 
Sample Data Grid 
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Figure 4 
Sample Program Plan 



Domain: Communication 
Decision Rule: less than 50% 
for two days (change plan ) 



Client/Child: 
type: 



Theresa 



Student: 



Group 



Individual 



Home 



Long Term Goal: Theresa will increase over baseline her spontaneous and 



appropriate use of signs accompanied with vocalizations to communicate her 
wants, needs and ideas. 



Short Term Objective #1: Theresa will imitate an adult model of a sign with 

i-XjiL vocalization with 90% accuracy over 2 consecutive days. 

• 
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Program Steps: 1. A: adult verbal and gestural mode l 



R: Theresa signs and vocalizes 



C: + give requested item and social praise 



- repeat model with physical assistance to sign 



Short Term Objective #2: 



Program Steps: 
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program with a developmental philosophy in which program staff intervene 
across adaptive behavior , gross-motor, fine-motor, language, and personal- 
social areas. However?, a program concentrating on the inclusion of parents 
in the assessment prooess and without the resources to hire, a psychologist or 
a qualified developmental specialist may find the Developmental Profile II 
II (Alpern & Shearer, 1980) more appropriate. 

If an instrument can be found that meets, or can be modified to meet, 
program-relevant criteria, serious consideration should he given to the test's 
adoption. A published or broadly disseminated instrument may have norms or 
psychometric information riot available for most "homemade" instruments. A 
more widely used instrument also may have been developed with more care, thus 
contributing to the instrument r s face validity and usefulness. And, widely 
used instruments may permit meaningful comparisons across children, groups, 
and programs. Since homemade tests are often poorly .developed and lack data 
on their psychometric properties, results can, be uncertain and suspect. How- 
ever, for populations that are severely multiply impaired, program staff may 
be required to develop instruments or make major modifications in existing 
measures. 

Level 3: Measuring Progress Toward Program Goals 

Assessing attainment of program goals can be used to monitor child 
change. Though, measurement of this type generally is more useful for moni- 
toring the more generic impact of the program on participating children and 
comparing, when possible , different approaches and programs. For level of 
assessment of progress and impact, adoption of a norm-ref erencejl or a stan- 
dardized test is the most desirable choice. The interventionist must decide 
the types of child progress goals, for all children or subgroups of children. 
These goals may be: 1) normal development in all areas, 2) normal development 
in several areas, 3) any development in all areas, or 4) any -development in 
some areas (May, 1979). If the program goal is normality in the sensorimotor 
domain, then the Bayley Scales of Infant Development (Bayley, 1969) may be an 
appropriate instrument to select. Because of their long-term summative 
nature, standardized tests are a wise choice when evaluating program goals and 
outcomes (Green, Ginsburg, & Hyman, 1974). However, programmatic evaluation 
poses problems similar to problems encountered with individual assessment. 
These problems include v?ide population variability, unsuitable instruments, 
and inappropriate design and analytic strategies (Bricker & Littman, 1982). 
Interventionists should be aware of the limitations of norm-referenced tests 
and appropriately qualify results (see Garwood, 1982; Ramey & Wasik, 1982). 

It is important to establish criteria for selecting an appropriate test. 
The test generally should have the following characteristics: 

• appropriate for the population, though the population on which the 
test was normed may be different 

• administered" and interpreted within the resources of the program 

• results useable, for but not necessarily limited to, program 
Revaluation 

• sensitive enough to detect change in children's performance 

• in reasonable agreement with program goals and objectives (Hamilton & 
Swan, 1981). 



14 14 



WHAT DESIGNS OR PROCEDURES SHOULD BE USED TC DETERMINE PROGRESS? 



Again, it is useful to use the three-level evaluation framework in Figure 
1 to examine appropriate designs and analytical procedures. Designs and 
analyses should vary according to the purpose, content, and level at which 
child progres s is being moni tored or program impact is being as ses sed • 

Level 1: Analysis of Daily or Weekly Progress 

At this level, the child's progress towards the acquisition and use of 
specific behaviors is of primary interest. Some form of daily or weekly 
information should be collected and then systematically compared with data 
collected previously. Data must be acquired in a consistent manner if the 
compai'isons are to be valid. A number of different strategies can be used. 
For example, the number of objectives reached within' a given period of time 
can be indicated. Data can be plotted on individual graphs to illustrate per- 
cent, proportion, frequency, or rate change over time. A single-subject . 
design is probably most appropriate for this level of analysis. Programs 
should attempt to demonstrate functional relat ions hips between the instruc- 
tional programs and changes In the child's responses or patterns of behavior. 
If resources needed to carry out effective reversal or multiple baseline' pro- 
cedures are unavailable programs should select a pragmatic way to examine and 
display the daily or weekly data to monitor child progress and to make sound 
educational decisions. - 

Level ^2: Analysis of Progress Toward Long;- and Short-Term Objectives 

The analysis described above (for LeVel 1) can be used at Level 2. If 
the children can be properly assigned to a Ifroup or subgroup, a group design 
approach would also be appropriate. Program staff have at least two general 
options for g'iroup analysis. The first involves a comparison of predicted 
progress and actual progress within the program. The predicted progress can 
be statistically compared to actual progress" in several ways, including refer- 
ence to existing norms, correlation between performance on pretest and post- 
test, and correlations between posttest of one year and posttest of the next 
year. Instead of a statistical comparison, expectancies or timelines for 
attainment of short -and long-term goals can be established. In this way, the 
child's actual progress can be plotted against the preset timelines for each 
long-and short-term target. These approaches require as sumpt ions of linear 
growth that are questioned in the professional literature. 

The second alternative requires the comparison of children in a par- 
ticular intervention program and children in other programs. This design has 
these problems: comparability of children across programs is difficult to 
ensure; critical program dimensions (e.g., staff) may vary; and assessments 
may favor one group over another (e.g., communication assessment that requires 
only an oral response may unduly penalize children in programs where alterna- 
tive communication strategies are encouraged). 



Currently, no perfect strategy exists for measuring child progress with 
groups of handicapped children (Bricker & Sheehan, 1981). Programs must eval- 
uate, their goals and resources and then select the design that offers the best 
compromise for .their particular population. 

Level 3: Analysis of Progress Toward Program Goals 

The analytic designs discussed for Level 2 are equally appropriate for 
this level of analysis. At "Level 3, some form of comparison seems mandatory 
if program impact is to be implied. A number of serious barriers exist t»hen 
evaluating progress toward program goals* These problems include wide popula- 
tion variability, unsuitable instruments, and inappropri ate designs and 
analytic strategies. 

Early intervention programs serve children with impairraJ. tits that range 
widely in nature and degree. Also, a wide range exists in tft* uducatiomil and 
socioeconomic characteristics of families involved in these programs* Often, 
this diversity allows only extremely small numbers of children to be mean- 
ingfully grouped. Heterogeneity affects the measures and designs that are 
applicable for examining program impact (Lewis & Wehren, 1982) • 

4 Existing instruments may be inppropriate for certain populations of hand- 
icapped children for at least two reasons. First, palectsd items may bf. com- 
pletely inappropriate for children with certain disabilities. Second, fuv 
more severely impaired children, the developmental space between items m/*y not 
be sufficient to reflect any change or growth. 

Most programs cannot randomly assign children to experimental acA control 
groups. Further, nonintervention controls are generally not possible for 
ethical reasons. The problems with the design options discussed in L&vex 2 
are also present in evaluating progress toward program goals (Bricker, 
Sheehan, & Littman, 1981). 

Barriers that face interventionists as they try to document program 
impact are significant and underlined with another pervasive difficulty. The 
majority of early intervention programs do not have enough resources to con- 
duct elaborate and controlled comparisons of program impact* This reality 
does not excuse interventionists who do not try to document program impact. 
Neither can poorly conceived and executed measurement plans be condoned « 
Rather, limited resources should require: 1) thoughtful compromise, 2) exten- 
sive planning to deploy limited resources most effectively, and 3) dedication 
to searching for acceptable alternatives to present designs and analytical 
procedures • 



Summar y 

Attempts to determine child change and program impact present enormous 
problems. Currently, the best we can do is to adopt those compromises that 
produce the greatest benefits. Limitations continue to exist in measuring 
child change and the impact of early intervention programs, though with the 
development of the field, appropriate solutions to evaluation problems are 
forthcoming. 
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Chapter 2 

Four Developmental Perspectives 

by Carl Dunst ~ 



WHAT IS CHILD PROGRESS ? 

A broad definition of child progress is 'actual, inferred, or perceived 
change in some specific or general dimension of child behavior.* In turn, 
change and progress imply that development of seize sort ha;3 occurred. An 
operational definition of progress and change depends upon one's philosophical 
and theoretical viewpoint (Baltes, Reese, & Nesselroade , 1977; Reese & Over- 
ton, 1970). 

Four major philosophical ideologies (Romanticisms Cultural Transmission, 
Progressivism, and Holism) and, respectively, four associated developmental 
perspectives (maturat ional , behavioral, process-oriented, and ecological) have 
had an impact on early intervention practices (see Ruast, 1981, 1982b; Kohl- 
berg & Mayer, 1972; Lambie, Bond, & Wsikart , 1975; Sevens & King, 1976) . 
Each ideology embraces a different definition of behavior change and develop- 
ment. Thus each ideology embraces a different way of defining what consti- 
tutes the most appropriate indices of progress. Table 1 summarizes the devel- 
opmental perspectives and definitions of change and progress derived from each 
of these ideologies. * 

Matu rat i onal 

This position views development as a progressive* unfolding of behavior 
which reflects maturation of the central nervous system (CNS). Change in the 
functioning of the CNS is inferred from age-related but not necessarily inter- 
related developmental landmarks or behavioral responses.. On a 10-item scale 
of reflex behavior, a child may on one occasion show only two reflex res^- 
ponses. If, on a second occasion, the child displays eight reflex behaviors, 
we may infer that a maturation of the CNS may have occurred. According to 
Gesseir, graded series of behavior landmarks are the measuring, rods or 
criterion of normal development. And the manifestation of these behaviors in 
response to standardized test situations* provides a basis for inferring the 
degree of maturity of the CNS. Similar concepts regarding changes in the CNS 
are espoused by Karl and Berta Bob&th in their neurodevelopmental theory and 
Jean Ayres in her theory bf sensory integration. Both perspectives view deve- 
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Tzble 1 

Comparison of the Four Major Developmental Perspectives 



Philosophical 
Ideology 



Developmental 
Fersptctive 



Definition of Development 



Major Proponents 



Romanticism 



Cultural Transmission 



Progressivisro 



Holism 



ik£aturational 



Behavioral 



Process-Oriented 



Ecological 



Unfolding of innate predispositions 
resulting from maturation of the 
central nervous system. 

Accumulation of learned behaviors 
as a result of reinforcement of the 
behavior. 

Progressive changes in the manner 
m which information is acquired, 
stored, and used. 



Differentiation in topography of 
behavior and adaptations to 
environmental demands. 



Arnold Gessell, Catherine 

Arnatruda, Carl and 

Berta Boboth, Jean Ayres \ 

B.F. Skinner, Albert Bandura, 
Sidney Bijou, Donald Baer, 
Todd Risley 

Jean Piaget, J. McVicker Hunt* 
In a Uzgiris, Robert McCaU, 
Kurt Fischer, Robby Case, 
Philip Zeltizo 

Urie Bronfenbrenner, Kurt Lewin, 
Moncrieff Cochran, Jane Brassard 



lopment as the increasing capacity to master control of the CNS; maturity of 
the CNS is reflected in so-callod normal patterns of movement. 



Behavioral 

According to this view, behavior is influenced by its environmental con- 
sequences; if behavior is followed by some reinforcing event, this strengthens 
the tendency to act or behave (Skinner, 1974). Behaviors which produ.ee rein- 
forcing consequences are called operants. The behavioral point of view con- 
siders behavior to be affected primarily by the environment. 

Development from a behavioral perspective is defined in terms of the 
accumulation of Earned behaviore: the more one learns, the more development 
has occurred. Change is most oftsn measured in terms of how many behaviors 
have been learned in a given period of instruction. 



Pr o ce s c-Ori e nt ed 

— This position views development as progressive changes in the manner or 
processes used to acquire, store, and use information. Changes in the parti- 
cular types of intellectual operations used to solve problems or adapt to moro 
complex demands are the principal indices used to measure developmental 
change. Piage,:*s theory of the progressive changes that (jeeur in the genesis 
of intellectual capabilities is perhaps* the most widely used process-oriented 
model for charge ufiri zing developmental change. According to Piaget (1970), 
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there are distinct levels or S£ag2S of development, each of which is charac- 
terized" by progressively more Sophisticated ways a child organizes, under- 
stands, and uses information uo respond to the environment. 

The \nformat ion -processing cheory (Zelazo, 1979) is another process- 
oriented concept of development and is beginning to influence early interven- 
tion practices. Development is < considered to be a progression of changes in 
higher-order strategies for acquiring, storing, and using bits of knowledge 
and information. 



Ecological 

This position views development as a progressive differentiation of a 
person's conception and knowledge of environments and the capacity to dis- 
cover, sustain, restructure, or alter environments at levels of similar or 
greater complexity (Bronf enbrenner, 1979). Development is considered to have 
occured if the form and content of a person !s activities, roles, and interper- 
sonal relationships become progressively more dif fe rent ia£ed v as a function of 
dirject and indirect experiences . Froa* an ecological perspective, progress and 
change are defined as differentiations and adaptations of behavior in response 
to demands and expectations of different persons and settings. According** to 
Bronf enbrenner (1979), a shift in "balance of power" away from adults toward 
the developing child is one major indicator of behavior change. For exampl^ , 
the child's capacity to initiate and sustain interactions with others as \ 
opposed to relying on adults to prompt behavior is one measure of a shift in ^ 
balance of power. 



; HOW CAN CHI LP PR OGRESS BE MEASURED ? 

A number of different strategies and approaches can be used to measure 
child progress and developmental change". Eac'i is derived from one or more of 
the developmental views described above. 

When we ask: "How can we measure child progress?" we really ask: "What 
Should be the outcome measure of the intervention effort?" An outcome measure 
(dependent variable) is something we expect lo change or affect as a result 
of some type of treatment (independent variable). Table 2 shows each of the 
major developmental views described in tiw? previous section and provides exam- 
ples of the methods of measurement and indices of behavior change associated 
with each view. 



Norm-Ref ere need Scales 



These standardized tests have been administered to large nrnbers of 
children for the purpose of determining the usual ages at which children reach 
different behavior landmarks. Standardized: norm-referenced instruments 



Table 2 

Methods Used to Measure Child Progress 



Developmental 
Perspective 



Maturational 



Behavioral 



Measurement 
Technique 



Norm-referenced scales 



Focus of Assessment 



In vivo observation/mea- 
surement of behavior; 
behavior rating scales; 
criterion-referenced scales 



Comparisons of child against 
performance of standardization 
Srt.:iipk& or standard established 
as expected outcome by the 
investigator. 



Determination of baseline perfor- 
mance and change in performance 
as a result of intervention. 



Examples of Indices of Progress 

Primary: Number of items passed, 
DA, MA, DQ, IQ, MDI 

Secondary: Gain scores; develop- 
mental rate scores; 
ratio of expected to 
actual progress . 

Changes in frequency, duration, 
rate, chaining of behavior; behavior 
engagement. 



Process-Oriented 



Ordinal scales; psycho- w 
physiological measurement 
of cognitive understand- 
ing; cognitive effectance 
and motivation 



Determination of the changes in 
cognitive performance resulting 
hor* challenging experiences 
(ii;c*!ivi* r .iion). 



Changes in problem-solving 
abilities; shifts in cognitive 
Capabilities; mastery of cognitive 
tasks; rates of progress. 



Ecological 



In vivo observation of 
behavior; social support 
rating scales; ratings of 
levels of independence; 
interaction ratings 



Deterrninatif;;! of dimct and 
indirect /n&uences that affect 
child pe rfonna&ee. 



Shift in balance of power; increased 
topography (differentiation) of be- 
havior, adaptations to environmental 
demands, parental perception/ex- 
pectations; situational outcomes 
(school pkcements, institutiona- 
lization, etc.) 



include the Bayley Scales, Gesell's Developmental Schedules 9 the Cat tell * 
Scale, the Griffith's Mental Development Scales, the Stanford -Einet Scales, 
the McCarthy Scales, and the Vineland Social Maturity Scale. 

Norm-referenced scales permit a comparison of a particular child 1 s per- 
formance and the performance of other children included in the standardization 
sample. The norms established for a given asses sifcGnfc instrument indicate a 
"typical" performance and, thus, provide a basitf for determining the extent to 
which a particular child's performance is adv^nc€*i ox delayed. 
■\ 

x 

Xprimaxy indices . Norm-referenced tests yield three primary indices of 
perf onhmice : ( 1) total number of times passed, (2) developmental age(s), and 
(3) development al quotients • 

The totaiNnumber of items passed on a test may be an appropriate measure 
of performance orMjr if there are an equal number of items at each age level.. 
If not, changes in\te number of items passed on different measurement 
occasions will not rerlect the same amount of prqgresjs for different children. 



Total number of items passed may be an appropriate measure for scales like the 
Griffiths and Cattell tests; it is not an appropriate measure for tests like 
the Bayley scales. 

Nearly all norm-referenced tests provide procedures for computing a deve- 
lopmental level of performance based on the total number of items passed by a 
child. This index provides a rough gauge for making statements like: "This 
child is functioning much like a child of T X T chronological age." A child's 
developmental age (DA) and mental age (MA) are af ten used to determine how 
much progress or change has! occurred between two separate measurement occa- 
sions. The child's DA or MA at a second measurement occasion is subtracted 
from that observed at a previous measurement occasion. 

A third index that can be derived from norm-referenced tests is a deve- 
lopment quotient (DO).* This index is computed as a ratio quotient or a devi- 
ation quotient. -- . . 

DO = Developmental Age x lQQ 
Chronological Age 

Deviation quotients are derived separately for different age groups of 
children in the standard sample. Tests that use deviation quotients always 
include tables for determining this index. 

Ratio quotients have different meaning than do deviation quotients and 
generally should not be used with scales that use deviation quotients for 
gauging rate of development. Typically, scales that use deviation quotients 
do not provide quotients below certain cutoff points (e.g., 50 on the Bayley 
scales); thus making them less useful for low-functioning children. In such 
instances one is tempted to use the test manual tables (for computation of 
deviation quotients above the cutoff point) and the ratio method to determine 
deviation quotients below the cutoff point . This should not be done. The 
best solution is, on an a priori basis, to use a test that will accurately 
characterize J)Qs for children functioning at quite varied developmental 
levels. Alternately, one could compute ratio quotient scores for all subjects' 
(if it is kept in mind that these indices are very rough measures of rate of 
development.) That is, if the children in a program perform both above arid 
below the cut-off point for a jgiven test, and one wants to have a common 
metric for contrasting rates of development, the child' s* mental age divided by 
his or her chronological age mulfcipled by 100 will give you a general indica- 
tion of each child's, performance. 

Derived indices . With regard to both the efficacy of intervention and 
measurement of child progress, this principal question is being asked: "Was 
child progress greater under conditions of intervention relative to a non- 
intervention condition?" Typically, we have some pretest scores and some 
posttest score on a group of children enrolled in an early intervention 
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Here DQ is used interchangeably with IQ, MDI, GQ, and other indices of 
performance that provide a metric of the relationship between actual and 
typical performance. 
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program. For illustrative purposes, we will assume we have data on a child to 
whom a developmental test was administered at 12 and 18 months of age. The 
child's developmental ages at those measurement occasions were, respectively, 
6 and 10 months, i From birth to age one year, the child received no interven- 
tion services. From age 12 to 18 months, the child was enrolled in an early 
intervention program. 

Computation of the child's ratio quotient for the two assessments shows 
that at age 12 and 18 months the child's developmental quotients were, respec- 
tively, 50 and 56. (This is a small and insignificant change if one considers 
the fact that the standard error of measurement for infant psychometric tests 
is generally 6 or more points.) Are we to conclude that . intervention produced 
no differential progress between tests? The answer is yes — if we,use 
primary developmental indices (i.e., developmental ages) as the outcome 
measures. However, this conclusion is flawed; the 18-month developmental age 
reflects experiences during the intervention period and the lack of experi- 
ences during the nonintervention period. A better way to assess progress 
during the intervention period is to separate changes due to the intervention 
period and the nonintervention period. Use the following formula to discern 
differential progress: 

(DA? - DAi ) (T)A|) 
(CA 2 - CAi) / TCAi) 

DAj and DA 2 are the child's developmental ages at the first and second 
measurement occasions, respectively . CA\ and CA 2 are the child's chronologi- 
cal ages at the respective measurement occasions. For our example, 

10-6 6 

18 - 12 / 12 = -67 / .50 - 1.34 gain 

Thus, this particular child made more progress during the intervention 
period than the nonintervention period, as shown by the ratio of expected 
(denominator term) to observed (numerator term) change.* ,We can conclude that 
something during the intervention period af fected the child' s development . 

The above formula is a derivative of the ratio deviation method for com- 
puting rate of development. This formula, however, separates out changes for 
the intervention and nonintervention periods. Similar indices of progress are 
- described by Bagnato and Neisworth (1980) and Simeonsson and Wiege rink (1975). 

Despite advantages over primary indices ,> the above derived index has one 
shortcoming. The computational methods are based on the assumption that nor- 
mal progress constitutes a unit increase in DA for every unit increase in CA 
(i.e., development is linear), and that acceleration of development equal to 
"normal" can be a reasonable goal for handicapped children. That is, a or*- 



A ratio score exceeding 1.0 shows that more than expected progres ; was 
made whereas a score less than unity indicates that amount of progress was 
less than during the nonintervention period. 
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month gain for each month in the program is expected under this assumption of 
normal development. Under most condit ions this is a tenable and useful 
assumption. However, with severely handicapped youngsters, this assumption is 
likely to result in a conclusion that significant change was not made when, in 
fact, it. was. An alternative is to use predicted progress based on th^ pre- 
test scores of a group., of ^handicapped youngsters (see especially Bricker, 
Sheehan, & Littman, I981)^rat her' than "normal" development , as the benchmark. 
Scores obtained by a group of, children at entry, into a program are used to 
predict scores at a later time. The discrepancy, between the predicted and 
actual scores provide a basis for gauging program, impact • The procedure 
involves the regression of DA on CA for pretest scores (scores obtained prior 
to intervention), the derivation of a regression equation to predict DA from 
CA, the computation of a predicted DA from the CA when post test was admini- 
stered, and a comparison of the actual DA with the predicted DA at this CA. 

Most early intervention prqgrams use standardized tests as one index of 
child progress. The above methods are approaches for computing progress from 
these scales. There are, however, other indices of development that might be 
used as measures of child progress. 

O 

Process-Oriented 

Ordinal scales . Ordinal, or Piaget ian-based scales, can be used to 
measure developmental change. These scales include series of items ranked by 
order of difficulty. Successively Higher levels of achievement represent 
progressively more complex forms of behavior in the genesis of dif ferent types 
of early cognitive competencies. For, example, the U^giris-Hunt (1975) scales 
measure performance in seven separate sensorimotor areas: object permanence, 
means-ends abilities, vocal imitation, gestural imitation, causality, spatial 
relationships, and schemes for relating to objects (play). Developmental 
status on ordinal scales is determined by noting the highest item passed 
(i.e*, landmark achieved) on each separate sensorimotor scale. 

Because there is considerable specificity in early sensorimotor develop- 
ment (Hunt, 1976), progress, and rate of development can vary between dif ferent" 
sensorimotor domains. Thus, if one wanted to assess the efficacy of vocal 
imitation training, progress along that particular scale could be compared to 
progress in other sensorimotor areas and differential effects of the training 
efforts could be determined. ■ '.."./■ 

Dunst and Gallagher (1983) recently described a strategy to use ordinal 
scales that has particular import for assessing the efficacy of early inter- 
vention efforts. The average ages at which successive ordinal landmarks are 
achieved for an intervention population are assessed. These mean ages and 
ages of infants in other samples for whom comparable data have already been 
collected are compared (see e.g.-, Hunt, 1976). Dunst (Dunst & Gallagher, 
1983) used this approach to show that a biologically impaired group of infants 
in a hdme-based early intervention program reached the top-level landmarks on 
an object permanence scale before biologically impaired and nonimpaired insti- 
tutionalized subjects. This suggested that intervention affected, in part, 
the patterns of development of the children. Other types of ^comparisons can 




also be made (e.g., age of acquisition of landmarks for infants entering 
intervention programs early vs. late; different types of intervention; and 
different levels of intervention). 

"Physiological measures . The use of heart rate as a prooess-oriented out- 
come measure is worthy of special note since it offers promise of being an 
index of change with children for whom psychometric, ordinal, and other overt 
behavior indices are inappropriate and discriminatory. Zelazo and his col- 
leagues (see e.g., Kearsley, 1979; Zelazo, 1979) demonstrated that heart 
•rate, when used as a measure of attention (together with visual fixation), 
often shows that development ally disabled children are not as delayed in cog- 
nitive understanding as the results of traditional approaches to assessment 
seem to indicate. The following paradigm is described by Zelazo: ^ 

The primary objective is to present an engaging visual or audito.ry 
event and to repeat that event until the child creates an expectancy 
for it. After a fixed number of presentations, and generally before 
the child has lost all interest in the event, a moderately discre- . 
pant variation of the standard is introduced and too is repeated. 
Following three presentations of the discrepancy, the standard is 
reintroduced for three presentations. • (pp. 239-240) . 

The child's capacity for creating, an expectancy for the standard assess- 
ment, whether or not the child recognized the reappearance of the standard 
following the discrepancy, and whether or not the child can assimilate the 
discrepant variation itself, are measured. 

Initial presentations of an event cause heart rate to decrease (signal- 
ling attention) but stabilize across repeated presentations of the same stimu- 
lus. Presentation of the discrepant event causes heart rate to decrease again 
if the child recognizes the events as being different. Zelazo has collected 
cross-sectional and longitudinal data on children between age 3 and 36 months 
which provide a basis for deciding "level of cognitive understanding • " 
Matching the responses of a disabled child to those of nondisabled children 
provides a basis for deciding the child's information processing capabilities. 
At present, this approach to assessment has been used only to discern level of 
cognitive integrity for determining what constitutes an appropriate "entry 
level" for intervention. Whether or not it might be used to assess child - 
progress is open to investigation but does seem that it has potential 
utility for this purpose, if it could be shown that following intervention the 
child's information processing capabilities showed advances relative to base- 
line performance. 

Behavior Measures 

Use of psychometric tests and, in some cases, ordinal scales assumes that 
an intervention is designed to affect the acquisition of the types of 
competencies that these scales measure. Often, this is not a good assumption. 
For example, for a program that is designed to af feet changes in the communi- 
cative abilities of young children, the Bayley scales, Uzgiris-Hunt scales, or 
any other test of infant intelligence would be an inappropriate outcome 
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measure. In such cases, measures more specific to the-goaiL of the intervene 
tion would be better indices of behavior change. 

The following five behavior indices can be used as outcome measure^;. when 
tailored to specific interventions: 

• Frequency — number of occurrences of a specific behavior d u r i ng a 
specified period of time (e.g., number of tantrums per hour)" 

• Duration — absolute amount of time spend engaged in .a specified! licKfe: 
vity (e.g., amount of time engaged in cooperative play) x . _ -^- : v~-^ 

• Proportion — differential amount of time spent in each of two Ot^more 
activities (e.g., time spent in tantrumls vs. time spenC in cooperative! 
Play) V *" 

• Rate — rate of change from one activity to another (e.g., number of _ 
changes from tantrums to cooperative play to tantrums) 

• Sequence — pattern of occurrence of two or more bfehaviors (e.g., 
tantrums, then cooperative play, then pushing, then crying, then 
tantrums , etc. ) 

~~ — • * • 
"Behavior engagement," (e.g., Bailey, Harms , & Clifford/ 1983; Montes & 
Risley, 1975) is a measure of the efficacy of early intervention which_has_ 
received increased attention and which deserves special comment because of 
its general usefulness. Engagement refers to the amount' of time a ctiild plays 
with materials, interacts with peers and adults, and otherwise remains 
actively involved with his or her erivironment in a development ally appropriate 
manner (Dunst, in press). This duration measure of behavior differs from the* 
typical frequency of production of behavior that has generally been used as 
the measure of child progress (e.g., Shearer & Shearer, 1972). To the extent 
that a child ic actively engaged witli his or her environment, opportunities 
abound for the possibility that the child will acquire Increased variatrion in 
his or her topography of behavior (role differentiation) and will learn to 
adapt to the demands of different settings, persons, contexts, etc. (Bronfen- 
brenner, 1979). Engagement would appear to be a particularly important 
measure of the efficacy of early intervention efforts if it could be demons- 
trated that engagement levels varied as a function of intervention efforts. 



Ecological-Based Outcome Measures 

An ecological perspective jpf-'tlevelopment , because of its orientation, 
suggests a number of additi^n^l outcome measures for assessing child progress. 
As noted by Bronf enbrermeif (1979) , early intervention programs have typically 
been evaluated using^er^ologically constricted outcome measures. This means 
that we have used^tfutcome measures that provide little basis for knowing about 
the child as he^or she functions in situations of everyday life. Moreover, 
Bronf enbrejfrrfer (1979) notes that we have almost exclusively focused on the 
child a^c he experimental subject with little or no attention paid to the 
interaction of chil^ and others — both animate and inanimate — in assessing 
6gress and behavior change. 

Shift in balance of power . According to Bronf enbrenner (1979), one index 
of development is the shift of power during child-adult interactions away from 
the adult toward the developing child. As a child acquires more socially 
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adaptive behaviors and becomes progressively better able to initiate and sus- 
tain interactions with others, one would expect the child to assert more con- 
trol over his or her environment, with a concomitant decrease in the need for 
an adult to do so for the child. This decisively simple contention suggests a 
relatively powerful means for assessing child progress. A measurement scale 
for determining developmental changes could be something as : simple as a list 
of "initiating behaviors" ranked by degree of difficulty with a four- or fiv^- 
point rating scale that allows one to assess the degree to which the behaviors 
are exhibited by the child and others within and across settings. Obtaining 
these measures across several months would provide a basis for determining the 
amount and type of shifts that occur in the balance of power between the child* 
and others. 

Role differentiation' . This outcome measure reflects the fact that deve- 
lopment involves increased variation in the types of bfehaviors acquired as a 
result of experience and the ability to adapt to the demands of different 
settings and contexts by either using what one already knows or constructing 
new behavior. Intervention efforts could define the types of differentiated 
roles expected to be acquired as a function of the intervention. (Construct a 
simple recording system that collects data on predicted and nonpredicted 
behavior outcomes, and compare the findings to see if expected changes 
exceeded changes for nonpredicted behaviors.) By tracking the topography and 
situation-specific manifestation of behaviors over time, the rate of occur- 
rence of behaviors can be plotted across situations to assess the net increase 
in behaviors that are learned as a result of involvement in an intervention 
program. 

Changes in perceptions of behaviors . According to Bronf enbrenner (1979), 
when developmental change occurs, it takes place concurrently in two domains 
— perceptive and active. I would argue strongly that some type of interven- 
tion may affect perceptions of a child's behavior. This , in turn, may affect 
actual behavior manifested toward the child. And this, in turn, is likely to 
affect child, behavior. A study conducted in the Child Development Laboratory 
at Western Carolina Center revealed that^parents with large degrees of social 
support available to them perceived their children's handicaps as- being less 
devastating than did parents who did not have aSL^t of support (Dunst, 
Trivet te, & Cross, 1984). Changes in perceptions may^he a "first order" 
effect of intervent ion efforts , and "second order" ef fedts include actual 
child change. 

Situational outcomes . Lazar and Darlington (1982) demonstrated that 
situational and contextual outcome measures may be good indicators of .the 
success of intervention efforts. For example, environmentally at-risk * 
children enrolled in early intervention projects were found to be less likely 
to be placed in special education classes or be retained in , grade than were 
at-risk children who were not enrolled in early intervention projects. These 
types of outcomes suggest a number of ecologically relevant measures of the 
efficacy of early intervent ion. Fdr example, Deno's (1970) cascade of ser- 
vices model suggests, an ordinal scale of possible placement options (regular 
class, special education class, home-bound, etc.) that could be used *as an 
ojut come "measure of the' effects of early intervent ion services • At Western 
Carolina Center, for instance, we have found that a handicapped child who did 
not rieceive early intervention services is twice as likely to be institu- 
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tion^lized as a child who did receive early intervention services. The use of 
"placement outcomes" as measures of program impact would provide considerable 
ecological validity to our evaluative efforts, and such measures are strongly 
indicated as indices of efficacy. 



WHAT ARE SOME STRATEGIES FOR DOCUMENTING 
THE EFFECTS OF EARLY INTERVENTION? 



The basic question being asked when evaluating the efficacy of interven- 
t ion is : "Did the intervent ion (independent variable) have the effect (depen- . 
dent variable) expected?" There are two broad types of efficacy studies: 
those that assess the impact of a specific intervention effort (e.g., a cup 
drinking training program) and those that assess the overall or general impact 
of an early intervention program. Strategies designed to assess the general 
effects of intervention efforts are described here. 

This section addresses three aspects of evaluative studies: defining the 
independent variable, asking the appropriate questions, and research design 
issues to answer evaluation questions. 

Defining the Independent Variable 

The independent variable is the thing that is done that is expected to 
produce some change or outcome (dependent measures) . Figure 1 (next page) 
shows the relationship betweien early /intervent ion as an independent variable 
and the outcomes (dependent variables )• that might be expected to be realized. 
The .independent variable is the intervention which is hypothesized to produce 
the types of changes described in the previous sections of this chapter. All 
of the indices, described as measures of child progress are the types of out- 
comes that one might hypothesize to be affected by the intervention, though 
good evaluation requires one to specify the exact types of changes anticipated 
as a result of a particular intervention. 

The preferred or recommended approach to establish the efficacy of a 
treatment or intervention is to compare children who received early interven- 
tion and those who did not. This requires an experimental design with a con- 
trol group where subjects are randomly assigned to intervention and noninter- 
vention conditions. However, the typical approach that has been taken is to 
use a pretest-post test design where an intervention population is tested at 
Time 1 and then again .at Time 2, apd the difference between the, measurement 
occasions is assumed attributable to the intervention (see Dunst & Rhein- 
grover, 1981). There are a number of assumptions of both designs that are not 
generally tenable, and thus make their use questionable for answering the 
efficacy question (see especially Dunst, in press). 

First, there is the assumption that all individuals who constitute the 
intervention populations benefit equally from tfe intervention. Second, there 
is the assumption that the amount of intervention provided the intervention 
population is similar for all individuals. Third, there is an assumption that 
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Figure 1 

Early Intervention and Expected Outcomes 
as Independent and 
Dependent Variables 
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defining the independent variables as involvement in aa early intervention 
program" accurately describes the events that affect child outcomes. 

I would argue that these as sumpt ions are of ten untenable and , £S a 
result, decrease the likelihood that benefits can be demonstrated e"ven when 
such benefits have in fact occurred. The above assumptions are factors that 
contribute to increased variability among subjects within groups, and it is 
this increased variability that makes it more difficult to statistically 
demonstrate the benefits of early intervention. As it turns out, we can take 
advantage of this variability in assessing the impact of early intervention. 
First, note the fallacies underlying the above three assumptions. 

The assumption about the homogeneous effects of early intervention . This * 
assumption supposes that most children benefit equally from early interven- 
tion. Efforts to document the efficacy of early intervention with biologi- 
cally impaired infants almost never take into consideration the probability 
that the intervention will have 'different impacts depending upon the levels of 
functioning, degree of retardation, or the handicapping conditions of the sub- 
jects. Thus, an outcome measure would be expected to show substantial vari- 
ability; this , in turn, decreases the probability of finding statistically 
significant results . 

The assumption about the intensity of intervention . This as sumpt ion 
supposes that the degree of involvement and intensity of intervention is the 
same for most program participants. This assumption is implicitly made 
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any time the intervention program, rather than specific dimensions of the 
program (e.g., classroom attendance, number of home visits), is used as the 
basis for defining the independent variable. Intensity of involvement varies 
considerably for a host of reasons, including parental level of interest, the 
child's propensity to bouts of illness, transportation problems, various 
family conditions, the child' s degree of impairment , etc. Since intensity has 
been found to be a predictor of progress (Sandow & Clarke, 1978), making the 
assumption about similarities in level of involvement seems untenable. 

The assumption about early intervention as an independent variable . This 
assumption supposes that the early intervention program is the principal or 
only intervention provided to the program participants. This assumption seems 
particularly untenable for a number of reasons. First, parents who seek out 
early intervention services are likely to access other services for their 
children (e.g., health-related activities) and themselves (e.g., parent sup- 
port groups). Second, social system theory strongly suggests that early in- 
tervention is only one of a number of support services (both informal and for- - 
mal) that are likely to have some impact on the child (Cochran & Brassard, 
1979) as* well as family functioning (Gabel, McDowell, & Cerreto., 1983). A 
number of implications are made when conceptualizing early intervention as the 
principal or only treatment provided to children and their families. Method- 
ologically, we may mistakenly at tribute changes on the dependent measures to 
the early intervention program when, in fact, changes were related to other 
support services or an interaction between the program services and other 
types of support. In fact, niany programs actively encoufage other support 
(both formal and informal) to children and. their families in addition to the 
services provided by the program. The failure to take this into consideration 
as part of defining "early intervention" as an independent variable is likely 
to reduce the probability of demonstrating program effectiveness. A better 
strategy for discerning the impact of intervention is to consider early inter- 
vention as one of a series of possible support services, to consider the vari- 
ous support services as an aggregate of "early intervention," and then to 
determine the relative contribut ions of this composite independent variable on 
the dependent measures of interest. This approach, described in more detail 
below, suggests a methodologically strong and ecologically realistic perspec- 
tive of the real -wo rid forms of support which have an impact on the child and 
family. * 

Alternative Ways of Conceptualizing the Independent Variable 

The problem with the assumption that interventions have no differential 
effects can be addressed by dividing the intervention population into sub- 
groups that decrease the variability within groups and thus increase the pos- 
sibility of at least demonstrating differential effects. Groups could be con- 
stituted on such variables as diagnosis, levels of mental retardation, early 
vs. late entry into the program, or any other variable that divides the group 
into subgroups for the purpose of assessing how group membership is related to 
some desired outcome. 

. The problem with the assumption that all individuals receive the same 
amount of intervention can be addressed by dividing the intervention popula- 
tion into subgroups according to degree, of involvement in. the program. For 
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example, in a center-based program, children could be grouped by days per week 
attending the program. Likewise, in a home-based program, number of home 
visits conducted during the year might be used to divide the group. 

The problem with the as sumpt ion that the intervention is the only or 
primary event affecting an outcome measure can be addressed by treating the 
intervention as only one of several independent variables that are related to 
the dependent variables of interest. For example^ a simple rating scale could 
be developed that measures the degree to which a child and his or her family 
receive a host of services from the intervent ion project and other agencies. 
By constructing the scale in a way that the children and families have equal 
access to the various services , either the total of the rating scores or the 
individual scores for separate services could be used as independent 
measures. 

An approach we have taken at the Family, Infant and Preschool PrGgram is 
to have the parents themselves define their level of involvement and satisfac- 
tion with various sources of support. This is .used as the independent vari- 
able. The scale consists of 13 potential sources of support to the families 
(Dunst, Jenkins, & Trivette, 1984). The parents rate, on a five-point scale, 
the degree to which each source has helped them care for and rear their handi- 
capped child. The total of the 18 Separate ratings and the ratings for the 
individual items are used as independent measures. 

There are a number of advantages for taking this approach to program 
evaluation. First, the approach is based on the sound assumption that 
parental ratings of the helpfulness and benefits derived from various sources 
is one way of defining the independent variable. Our own research supports 
this assertion. In several studies we have conducted, amount of support 
available to families of handicapped youngsters has been four^d to be signifi- 
cantly related to a number of parental, family, and child [outcome measures. 
Second, if different programs use the same scale, the programs* could combine 
data across projects to build a sufficient sample to perform & variety of 
statistical tests which otherwise could not be performed ~~ a problem that has 
plagued early intervention efficacy studies (see Dunst *Jt Rhelngrover, i981}o* 

Posing Appropriate Questions - 

The reader should note that the above strategies not ask the question: 
"Does early intervention work?" Rather, the strategies are designed to answer 
the question: "What particular aspects of interventions and other fornULof 
support affect program outcomes?" Note 'also that the latter question is 
designed to address issues regarding the specificity of intervent ion effects 
and the unique contributions different variables make to different outcomes. 
Evaluative efforts that answer this question provide specific -information 
regarding what does and does not affect program outcomes. This information 
can demonstrate the efficacy of intervention efforts and can be useful when 
justifying improvements and. changes in program operations. 

9 Posing the proper question in terms of the relationship between indepen- 
dent and dependent variables is also an important part of evaluat ive s tudies . 
Generally, evaluation research in the field of early intervention has been 



conducted in an atheoret ical manner*' This is .the case becau<u> the relation- 
ship between what one does" (independent variable) and what %one expects to 
occur (dependent variable) has not been well conceptualized. For the most 
part, the field of early intervention has not adequately defined whaT consti- 
tutes "early intervention." Further, and as a result of a failure to define 
the independent variable, outcome measures often assess competencies that may 
or may not be assumed to be affected by what one does amder the rubric of 
early intervention. For example, most early intervention urograms use stan- 
dardized instruments like the Bayley scales or nonstandard*. zed instruments 
like the Early LAP (see e.g., Garwood, 1982). If so m&ny projects are 
designed to develop innovative efforts, why do most of theme projects use the 
same or similar outcome measures!? Better conceptualized studies are sorely 
needed if we are to begin to unravel what it is that affects, different program 
outcomes • 



Several Generally Feasible Research Strategies 

The types of questions just posed do not reculte nontn&atment control 
groups to evaluate, the effects of intervention Because the key question 
asked is "-What aspects of early intervention affect program outcomes? 11 no 
control groups are needed and yet the influence of specific variables can be 
determined. Two generally feasible approaches for doing so are described 
below. 

Split-group design . In those instances where an intervention population 
can be divided into two or three s\ibgroups on the basis of some important 
explainer variable,: a comparison can be. made of the differential effects of 
intensitv of treatment, level of inyolv erne nc 7 parental satis faction,, etc. For 
example, if level of intensity were predicted to affect program outcomes, and 
the participants in the program varied according to the amount and time of 
services they received, a median split of the group into low- and high- 
intensity subgroups could be made. Then the t#o subgroups could be compared 
to determine if in fact the high-intensity group made significantly greater 
gains , changes , etc. If this were confirmed, we would then have evidence to 
suggest that level of intensity was a significant contributor to change on the 
dependent measures. 

The split-group design has general utility as long as the method of 
grouping subjects on the independent variable is based entirely on the type of 
predicted relationship between the independent and dependent measures. The 
split-group design can be used whenever one can conceptually predict such 
relationships. For example, at the Western Carolina Center, an ecological- 
social support model is used to structure intervention efforts. The model 
predicts that social support available to families affects parental well- 
being, family integrity, styles of interaction between parents and" children, 
and selected aspects of child behavior. Parents complete a simple social- 
support rating scale which is used to group the parents according to low, 
median, or high degrees of support. Group membership is then related ±o the 
above classes .of dependent measures (Dunst , 1982a) .~ This strategy has allowed 
us to identify a number of different and revealing relationships between 
social support and the outcomes which are expected to he influenced by 
support. 




Mult iple regression analysts . In those instances where we have a series 
of independent measures and we want to determine the unique contributions o* 
each in terms of affect (for example, child outcomes) a multiple regression 
analysis esn help (see especially Cohen & Cohen, 1983). Multiple regression 
is a general method for assessing the -relative contributions of a serio* of 
different independent measures to one dependent meaure. Two approaches v"# 
regression analysis seem to have utility for the field of early intervent ion. 

Method 1. Assume that we have a relatively distinct set of variables} 
that <w predict have some affect on a specific child outcome* The predictor 
or independent variables might be child sex, number of individual activities 
prescribed,, precentage of c home visits made, ratings of -parental commitment to 
the interventions, etc. The strategy to determine the relationships between 
each predictor variable and the child outcome measure involves performing a 
multiple regression analysis where child outcomes (Y) is predicted from bt-*. 
optJriil linear estimates of combinations of the independent variables (Xj, X?, 
X3. ICft)* The analysis can proceed in one of three ways. Fir&t, a full 
regression model can be used where all the independent variables ar# simul- 
taneously regressed on the dependent measure. A signif leant result would 
indicate that the combination of predictors affect the outcome in a manner 
which suggests a casual or mediational' relationship between the independent 
and dependent variables. Second, a stepwise regression analysis can be per- 
formed. In this approach, the independent variables are entered one at a time 
until ixo independent variable accounts for a significant: amount of variance in* 
the dependent variables. The order of entry provides a basis for discerning ' 
the relative contribution of the independent variables. The tadeper*de:at vari- 
able most related to the dependent variable is entered firsts the next most 
related independent variable is entered second, and so on. Thiro % a hier- 
archical regression analysis can be used. In this method, the ot &r of entry 
of independent variables into the equation is determined on an a 1 riori basis 
and dictated by the nature of the evaluation effort. At each stejr in the 
analysis, the increments (I) in R2 are determined and the significance of the •* 
variance accounted by I is determined. Thus, the extent to which specific 
independent variables are related to the dependent variable and the speci- 
fic independent variables that are important predictor variables in terms of 
affecting changes in the dependent variable r- can be assessed. 

Method 2. The second method for assessing the contributions of a series 
of independent variables is to use multiple regression analysis where the 
independent measures are conceptually or prpgrammat ically similar. - For exam- 
ple, the independent meausures might be ratings of the degree to which parents 
find, for example, that each of 10 sources of support helps them care for 
their child. Or, staff or parents might rate the extent to which children are 
involved in a number of different services offered by a program. The simul- 
taneous, stepwise, or hierarchical models can be used to discern the relation- 
ship between the independent variables and the dependent variable, .although 
only the hierarchical approach can tell which specific independent variables 
/are most important. Thus, the hierarchical approach is recommended. The 
reader is referred to Dunst (in press) and Dunst & Trivet te (1984) for more t 
in-depth discussions and illustrations of this approach to program 
evaluation. 
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CONCLUSION 



The extent to which the efficacy of early inte mention can be established 
in terms of its impact upon child progress depends upon !iow the relation 

ship between the intervention (independent variable) &nd cmLcome Measures 
(dependent variable) are specified. Especially crucial for good program 
evaluation are adequate definitions of the independent variable , recognition 
of the assumptions inherent in one's conceptualization of the '.inter/exit ion, a 
selection of outcome measures that one would expect to be ef fect ** by the 
intervention, and methods for discerning " the impact of the "iftfce?.ven£ tons • 

It is hoped that the various discussions presented! lore stimulate the 
reader to better design program evaluation studies- By doing ?o, the effects 
of early intervention on child progress can be accurately discerned.* 
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Chapter 3 

Sources of Difficulty 



by Nanqj Johnson-Martin 




WHAT IS CHILD PROGRESS? 



Most interventionists probably agree that the measurement of child prog- 
ress is critical both for planning appropriate interventions for individual 
children over time and for assessing the effectiveness of the programs that 
provide intervention services* Little agreement exists, however - 9 when decid- 
ing what to include in measures of child progress for these two purposes* in 
defining child progress within this qontext, at least three sources of diffi- 
culty exist: 1) the discrepancy between implicit and explicit goals in prog- 
rams serving handicapped infants; 2) the diversity of the needs of children 
served in early intervention programs; and 3) the different kinds of training 
of professionals who staff early intervention programs* ' 



Implicit vs* Explicit Goals 

Most early intervention programs define developmental progress in terms 
of the number of developmental milestones achieved by children over a given 
period of. time* Thus, the explicit statement of goals primarily involves in- 
creasing the rate of developmental progress of these youngsters* A visit to 
any intervention program, however , would convince an observer that skill 
mastery is not the sole focus of early intervention* Parents are consoled, 
counseled, and trained; time is taken simply to entertain or play with the 
children; adaptive equipment is developed arid modified* Children sometimes 
appear to be ignored as parents and interventionists discuss the meaning of 
certain responses, ways to deal with baby sitters, how to qualify for food 
stamps, etc* The time spent in these activities is not wasted* Rather, this 
time is used by staff and parents to meet implicit goals for themselves and 
for the children. 

To assess all aspects of child progress related to an intervention pro- 
gram, one must consider the way time is spent within the program and the pos- 
sible effects on the children* For example, time spent training -and counsel- 
ing parents is often directed at helping the parents feel competent to deal 
with their children and to interact with them in a reciprocal fashion* Though : 
the focus is on the parents, the children might be expected to change as a re- . 
suit of interactions with the parents* 
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The definition of child progress should be broad enough to include the 
changes in children that occur as a result of better parenting or as a result 
of a generally less stressful environment. Items should be included in prog^- 
ress measures that reflect how the child is evolving as a member of his or her 
family and community. Such a view presents difficult measurement problems 
but, in the long run, will be more effective than methods now used to describe 
the progress of children in early intervention programs. 

» ■ ■"* 

Needs of Children 

o ' 

Some of the difficulty in assessing progress in early intervention pro- 
grams stems from the history of the early intervention movement. The first 
targets of intervention were children who Were considered to be deprived of 
adequate stimulation and who could, therefore, benefit from an education or 
conpensatory stimulation program. Some of. these children were born into 
impoverished homes, and some f spent their early months in sterile hospital 
environments because of prematurity or other complications of pregnancy or 
delivery.. For these children, a stimulation model was appropriate and child 
progress could reasonably be described by how clearly the child's development 
paralleled that of his or her nondeprived peers. When children with clearly 
atypical patterns of development became the targets of early intervention, 
however, this strategy often proved inadequate. These children needed more 
-than stimulation; they needed active treatment or therapy. 

In some instances, the effect of intervention might not be the accelera- 
tion of development; it may be the prevention of secondary handicapping condi- 
tions. For some of these children, "normal" interaction with the environment 
will never be possible. The focus of intervention may be to teach them 
.atypical but functional ways to interact with the world (e.g., using manual 
signs or a communication board instead of, or in addition to, speech). 

Unfortunately, the measurement techniques found useful with children at 
risk for developmental problems because of environmental ^factors were adopted 
somewhat uncritically by programs serving children with handicaps due to bio- 
logical factors. The Intelligence Quotient (IQ) or Me nt al- Development Index 
(MDI) fixation has plagued early intervention programs by focusing attention 
on meaisures that may be reliable and scientifically respectable for some chil- 
dren, but insensitive to the progress made by atypical children. 



Professional Training 

In the beginning of the early intervention movement for biologically im- 
paired children, programs tended to have an educational or a. medical orienta- 
tion. The basic- goal was to promote developmental change, and the training 
and philosophical stance of the service providers, rather than the nature of 
the children, were often co^sidere^d to be more important determinants of the 
way this goals was defined knd implemented. Programs with an educational 
orientation focused attention on learning and behavior. Developmental tasks 
were analyzed and each step was taught with appropriate reinforcement. Pro- 
grams with a medical orientation were more concerned with underlying 
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neurological development, the quality of movement patterns, and" the concept of 
"readiness" for mastering tasks. 

Most intervention programs today. adopt a much greater integration of 
these two points of view, though some of the problems in defining progress 
stem directly from the philosophical and treatment issues that separate thera- 
pists from behavior! sts. These issues are rarely divisive for children who 
are mildly to moderately impaired and whose development will follow typical 
patterns (though, perhaps, at a slower rate). Problems surface when the chil- 
dren are markedly atypical. The behavioris t f s tendency to describe progress 
in terms of steps toward mastery of a specific task may be antithetical to the 
therapist's effort to build on underlying skills. For example, when teaching 
a child to sit alone, the behavioris t may attempt to place the child in a sit- 
ting position, propping ao much as necessary, then gradually removing the 
props and measuring progress by how long the child is able to remain upright. 
The therapist, however, may .not place the child in a sitting position at all. 
Rather, the therapist may work on a variety of activities in the prone posi- 
tion to strengthen trunk muscles that will later allow the chil'd to sit unsup- 
ported. Progress toward sitting may be measured by how well the child masters 
the skills necessary to reach a sitting position unassisted. 

When working with -the most severely handicapped children, the . goals of 
the therapist and the behaviorist may differ even more sharply. The behav- 
iorist wrill tend to set goals that indicate skill mastery. r The therapist may 
set goals that are primarily related to avoiding muscle contractures, main- 
taining range of motion, maintaining body functions, etc. From the thera- 
pist's standpoint, "not getting worse" can be considered progress — a concept 
difficult for the behaviorist to accept and justify. •■ 

/ Before program staff can select instruments or techniques for measuring 
Child progress, they must first examine their program and reach some thought- 
ful agreement about its goals and objectives. The implicit goals. (goals im- 
plied by the way time is actually spent) should be made as explicit as possi- 
ble. The next step is considering how children in the program might be. af- 
fected by. the goals that focus directly on them and by the goals that focus on 
their families. The program staff will then be ready to develop a definition 
of child progress that is broad enough to reflect both the unique characteris- 
tics of the .children served and the activities that are a part of the inter- 
vention program. With this definition in mind, measurement instruments and 
techniques can be selected that are appropriate for assessing child progress 
within the context of that program. ^ 



HOW CAN CHILD PROGRESS BE MEASURED? 



As noted earlier, the primary purposes for measuring child progress are 
to track the development of individual children so appropriate interventions, 
may be planned and to assess program effectiveness. Each of these purposes 
requires appropriate measurement instruments. 
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Tracking the Progress of Individual Children 



When- the^ purpose of assessment is tracking a child's progress to plan 
^.appropriate interventions, the measurement instruments chosen should not be 
biased against the child's particular handicaps; the instruments should also 
reflect the .desired content of intervention. Several instruments have been 
designed specifically for children who are sens Qrily impaired and cannot be 
assessed with. standard instrument s (e.g., the Reynel 1-Zinkin Scales— -1979 — 
for Young Visually Handicapped Children) • Although sucti tests generally have 
poor norms, they are appropriate for tracking the progress of individual chil- ; 
dren because they are based on a knowledge of the " normal" development of atyp- 
ical populations. For children with severe motor impairments , no such instru- 
ments exist. There' are, however, a f&w criterion-referenced instruments that 
may be modified for use with children having various physical limitations 
(e.g., the Assessment Log of the Carolina Curriculum for Handicapped Infants— 
Johnson-Martin, Jens, & Attermeier, in press). * ~ ..^ ; :7.V. J : ' 

Children with multiple handicaps present special assessment problems*. 
The clinician often must choose items from a variety of instruments and modify, 
them to accomodate the child's particular handicapping condition* This type 
of modification invalidates the test norms; however, if the goal is simply to* 
chart child progress, such invalidation is not critical. 

Assessing Program Effectiveness- 

Assessment for the purpose of documenting program impact on child devel- 
opment involves much more than simply charting child progress. For almost all 
children, some development al progress will be made regardless of whether any : 
intervention has been provided. When do cument ing pr ogres s |that results from , 
intervention, total progress must be separated into two components: 1) prog- 
ress due to development and 2) additional progress due to interaction between 
the intervention and the underlying developmental process. ■■ '.' " 

When instruments are chosen to document intervent ion --related progress, 
many factors should be kept in mind. For example, what research design will 
be used to document intervent ion effectiveness? Will the design be one in 
which two groups of children are compared, one in which the progress of a 
group of children is compared against some standard, or one in which interven- 
tion effectiveness is documented in individual children and them summarized 
for the group (a single-subject design)?^ Additionally/ if groups of children 
are to be compared, what are the characteristics of the children in each 
group? Are they relatively homogeneous (e.g., all have Downls syndrome) or ^ 
heterogeneous? Is any one instrument appropriate for all the children? What 
instruments (if using a Single-subject design) will lend themselves to com- 
bining data from different children to document group progress? Issues such 
as these should be considered carefully before an assessment Iplan is 
developed. 
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TYPES OF ASSESSMENT INSTRUMENTS 



Nonn-Ref erenced Assessment 

An important goal of intervention programs is to accelerate the develop- 
mental progress of the children they serve. Showing increases in child IQs 
or MDIs Is one way to document this acceleration. Norm-referenced instruments 
such as the Bayley Scales of Infant Development (Bayley, 1969) and the tfattell 
Infant .Intelligence Scale (Cat tell', 1960) are diagnostic instruments that 
identify those children who are developing normally, those who are developing 
slowly, and those who are developing at an accelerated rate. Norm-referenced 
instruments must be administered i*ua stardardized fashion by trained examin- 
ers using materials provided by the test developers. A child's performance 
is compared to that of other children 0 in the normative sample. Each child's 
. status is described in terms o'f deviation from the mean of children of conr- 
parable age (MDI or 10), or in terms of the age at which normally developing 
children accomplish the same sort of tasks (developmental age or mental age). 
These tests, which have'been widely used to as ses a child fpr ogress in interven- 
tion programs , are relatively reliable and the MDIs or »IQs are presumed to * 
reflect rate of development. A child who obtains a near 50 IQ on several lon- 
gitudinal assessments is progressing at about half the rate of normally devel- 
oping children. This child is maintaining his or her developmental rate. On 
the other hand, a child who .obtains an IQ of 50 on one evaluation and an IQ of 
60 on, a subsequent evaluation is considered to be developing at an accelerated 
rate. 

However, using infant tests as rate measures has been questioned by sev- 
eral authors because 1) infant development appears discontinuous across devel- 
opmental stages' (McCall, 1979) 5 2) the kinds of skills included. in the tests 
vary widely from one age level to another (Johnson, 1982)* and 3) these tests 
are rarely sensitive to change in significantly handicapped children (sensory 
or motor impairments in these children preclude "normal" interaction's with the 
test materials* under standardized conditions). 

Cri t e r i on-Re fere nee d As se s sme nt 

Criterion-referenced instruments assess current skills, identify instruc- 
tional objectives and, -when used longitudinally, assess progress in meeting 
instructional objectives. In theory, a criterion-referenced test is developed 
by first breaking important skills into their component parts, then'into the 
separate steps necessary for mastering each part . A test item is developed to 
match each step, specifying what will be learned as well as the criteria by 
which task mastery will be judged. 

With criterion-referenced assessment, a child is described by the tasks 
that liave been mastered, not be comparisons with o.th^r children. The tasks 
not yet mastered become instructional goals, beginning with the easiest tasks 
and progressing to the more difficult ones. Progress is documented by the 
number of new tasks mastered between as ses sme nt points • ' 

/ 

For infants , cri terion-iref erenced tests are problematic because of the 
difficulty in identifying appropriate test content. Little is known about 



which infant skills are related to critical childhood skills (reading readi- 
ness, for example)* In addition, little information exists regarding which 
infant skills are most amenable to training and which are determined primarily 
by neurological maturation. Most criterion-referenced instruments for infants 
have been developed not on the" basis of a theory~nor on the basis* of identify- 
ing prerequisite skills for later adaptive behavior and academic achievement. 
Rather, these instruments have been developed by adopting items from various 
norm-referenced tests, then grouping these items into domains (according to 
item content), and arranging them according to the mean age at which normal 
children' pass the items. 

Assessing progress that results from intervention is difficult when using 
criterion-referenced tests. These instruments include items inappropriate for 
seriously handicapped infants; in addition, no standards exist to indicate how^ 
many items would have been mastered simply through maturation. . To deal with 
this problem, many instruments assign developmental ages to the test items, 
then compute ratio scores (developmental age divided by chronological age, 
multiplied by 100) for each domain. If the ratio increases during the inter- 
vention period, then it is assumed that progress results ffcm intervention. 
This assessment procedure contains many "methodological problems (Johnson, 
1982), among these: 1) developmental ages assigned to each test Item repre- 
sent the mean age at which normally developing children passed the item, but. 
do not reflect the variability of children in mastering the skill; 2) the 
steps between each itejn reflect unequal intervals of difficulty; and 3) the 
items have been drawn from a variety of developmental tests representing dif- 
ferent normative samples. 

One way of resolving these methodological problems is to dispense with 
the notion of ^developmental ratios (White, 1979), and instead, compute the 
percentage of < test items passed in each domain at each assessment point. Per- 
centages for children "who are receiving intervention and those who are not can 
then be compared. Or, these percentages can be compared between domains tar- 
geted and those not targeted during a particular intervention. Percentages in 
all domains can be compared between programs providing different forms of 
intervention. Despite the problems with this procedure, it remains a sound 
method of documenting progress due to intervention and is worthy of further 
consideration. 

- ' 3 

Ordinal Assessment 

Ordinal assessment refers to instruments that arrange the component 
skills of long-range objectives according to degree of difficulty. Existing 
infant tests that claim to be ordinal tests are based on Piagetian theory 
(e.g., Uzgiris & Hunt, 1975). Piaget's theory defines seven domains of cog- 
nitive development, as well as six developmental stages that occur within each 
domain during the sensorimotor period (birth to approximately two years of 
age). The assessment includes items that will identify the stage of a child's 
development Within each domain. 

Although not intended as instructional guides, ordinal tests for infants 
have been used as criterion-referenced .tests in infant education. Since these 
tests focus primarily on cognitive and language development , it is necessary 
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to choose additional instruments for assessing the broader objectives of most 
intervention programs. Although final goals and the steps for reaching them 
are better defined ; problems still arise in documenting the progress related 
to intervention. As with criterion-referenced tests , it is difficult to de- 
termine how much progress is due to maturation and how much to training. In 
addition, the cognitive items contain heavy sensory and motor requirements, 
making it difficult to assess the progress of many handicapped infants. 



Nontraditional Assessment ' 

Nontraditional assessment is any procedure, formal or informal, which 
gives information that may be unavailable from published norm-referenced or 
criterion-referenced tests. A renewed interest has emeijet* recently in find- 
ing reliable ways to assess cognitive skills in children whose physical handi- 
caps or behavioral patterns prevent use of traditional assessment procedures. 
Among the nontraditional procedures are: 1) a procedure for assessing dis- 
crimination and memory in children functioning under six months of age ("Fagan, 
1975); 2) a procedure for determining whether a child can learn a sequence of 
events and can recognize an alteration of that sequence (Kagan, Kearsley, & 
Zelazo, 1978); 3) a systematic presentation of items that tetrf to elicit smil- 
ing .or^lughter at different ages in infants (Cicchetti & Sroufe, 1976), and . 
4) a va|igty of operant learning procedures that assess a child's ability to 
learn how to produce an interesting environmental event (e.g., Brinker, 1981; 
Goldman, 1980). 

These nontraditional procedures correlate with other, more traditional 
measures of cognitive development while making very limited demands on the 
child's motor competency. Nontraditional instruments are able to identify 
cognitive competence not always evident in other assessment procedures. .In 
non-traditional assessment, child progress may be observed by repeated 
measures; however, it would be difficult to .demonstrate that progress is due 
to intervention in any but the operant learning procedure. 



THE MULTIVARIATE ASSESSMENT APPROACH 



Simeonsson, Huntington, and Parse (1980) have argued for a broader, child 
assessment than that which is offered by the assessment procedures normally 
used in intervention programs . They suggest using a variety of instruments to 
assess child progress since the instruments sensitive to changes in one handi- 
capped child may not be sensitive to changes in another child. In addition to 
traditional measures of development they suggest including measures of 
adaptive behavior and temperament in. tests that assess the progress of signi- 
ficantly handicapped infants,. 



Adaptive Behavioif 

The definition of adaptive behavior poses interesting problems in the 
area of infancy. Traditional measures (e.g., the Vineland Social Maturities 
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Scale, Doll, 1965; the Preschool Attainment Record, Doll, 1966) include few 
test items for the infancy period; also, these items emphasize motoric behav- 
iors. Little emphasis is ^placed on the nonmotoric components of social and 
communicative behaviors often considered important components of adaptation in 
infancy. The new revisions of the Vineland Social Maturities Scale (i.e., the 
Vineland Adaptive Behavior Scales, Sparrow, Balla & Cicchetti, 1984), provides 
more items for the infancy period and includes four domains: communication, 
socialization, daily living skills, and motor skills. The Vineland Adaptive 
Behavior Scales may provide a better measure of adaptive behavior in the in- 
fancy period than has been available previously. However, no reports of its 
use with very young children have yet been published. 

Another promising instrument Is the Carolina Record of Individual 
Behavior (Simeonsson, Huntington, Short, & Ware, 1982). The CRIB, which ex- 
pands the Infant Behavior Record used with the Bay ley Infant Scales (Bayley, 
1969), is sensitive to changes in severely handicapped children for whom the 
usual developmental measures are insensitive. The CRIB is also responsive to 
changes in children who have specific sensory and motor impairments - 
(Simmeonsson et al., 1982). 



Temperament Measures 

Most temperament measures used with infants (e.g., Revised Infant Temper- 
ament Questionnaire, Carey & McDevitt, 1978; Infant Behavior Questionnaire, 
Rothbart, 1981) depend on the judgements of parents or other caretakers 
regarding the characteristics of the infants. Because of this, temperament 
measures are criticized as being more a measure of the adults that fill them 
out, rather than an objective measure of infant characteristics. Such 
measures may be seriously considered as measures of child progress. However, 
if one adopts a transactional view of development, then such measures may be 
considered for assessing child progress. In the transactional view, it is as- 
sumed that the development of family members is affected by their interactions 
with each other. If a parent reports changes in a child's temperament, it may 
be assumed that changes have occurred in the family system which are impor- 
tant in the child's development. Several possibilities exist: the child may 
have changed, thus changing parent perception; the child actually may have 
changed little, but altered parental perceptions may create changes in the 
child; or both parent and child may be changing in a mutually reinforcing way. 
In any event, temperament measures do reflect child progress. 



An Appropriate Test Battery 

When choosing a multivariate assessment approach, the purposes for 
assessing progress as well as the characteristics of the children to be 
assessed must be. considered carefully. If a program is serving a hetero- 
geneous population of handicapped youngsters, the best- strategy for docu- 
menting individual child progress may be to tailor as.segsments individually 
for each child, choosing instruments appropriate to their abilities and disa- 
bilities. Once this choice is made, however, it will be impossible to use 
your group data with traditional research designs to demonstrate program ef- 
fectiveness. . _ . 
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Another strategy for documenting progress is to choose one comprehensive 
criterion-referenced instrument for use with all the children. This instru- 
ment would be one or more instruments sensitive to behavioral and tempera- 
mental characteristics in all the children. Finally, a variety of other in- 
struments tailored ta-meet the needs of individual children would be. chosen. 
For example, one might choose the Vulpe Assessment Battery (Vulpe, 1977) , the 
Hawaii Early Learning Profile (Furuno, O'Reilly, Hosaka, Inatsuka, Allman, & 
Zeisloft, 1979), or the Assessment Log of the Carolina Curriculum for Handi- 
capped Children (Johnson-Martin, et al., in press); these could be supple- 
mented with the CRIB (Simeonsson et al., 1982) and the Revised Infant Tempera- 
ment Questionnaire, (Carey & McDevitt, 1978). Then, some nontraditional tech- 
niques could be used to assess cognition in the more severely motor-impaired 
children, along with the Reynell-Zinkitt- ( 1979) for use with children who are 
visually impaired. In addition, the Parenting Stress Index (Abindin , 1983) 
might be used to determine how much stress the infants exert on their fami- 
lies. (From a transactional viewpoint, a decrease in family stress would be 
one additional indicator of child progress.) 



WHAT PROCEDURES SHOULD BE USED TO DETERMINE IF PROGRESS HAS BEEN MADE? 



The standard scientific method for demon sit rating the effectiveness of any 
treatment is to assign subjects randomly to either a treatment group u * 
control group, or to a group that receives treatment A and a group" ttv re- 
ceives treatment B. 'Pre- and posttestin*. will determine whether the groups 
have made different degrees of progress or change resulting- f rora the treatr 
ments. Obviously, this design is problematic in the area of early interven- 
tion. Ethical considerations prohibit assigning children to a control or no- 
treatment group. Some programs have used existing conditions to by-pass the 
ethical problems, e.g., by assessing the progress of children on a waiting 
list and comparing it to the progress of children already in a treatment pro- . 
gram (this is not random assignment; and the comparability of the groups is a 
concern). Although a "treatment A vs treatment B" design may be more ethical- 
ly sound, difficulties do exist in identifying different treatments and in en- 
suring the comparability of the groups. Some studies compare the progress of 
children in an intervention program with children whose development is, re- 
ported elsewhere in the li terature (for example, Hanson's, 197 7, comparison of 
the children in her Down's syndrome intervention program with data available 
on the development of home-reared Down's syndrome children not in early inter- 
vention programs). While appealing, such comparisons are fraught with meth- 
odological problems related to comparability of the groups. 

In attempting to document intervention effectiveness, many prgrams have . 
used, either a standardized or a criterion-referenced test to which 
developmental levels were attached for demonstrating developmental rate 
changes within individuals and within the group. 



Alternatives tfr Experimental and. Control Group Designs 

New methods of documenting intervention effectiveness mus ? t be found if 
early intervention programs are to receive continued support. A number of 
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intervention programs are considering some viable alternatives to group de- 
sign^ These alternatives are:' 1) multiple baseline design, 2) a single- 
subject design comparing the longitudinal development of a child in interven- 
tion with a normative sample, and 3) joal~at tainment scaling designs. 

Multiple Baseline Designs 

Developed to assess behavioral changes, multiple baseline designs involve 
measuring several behaviors at intervals until their stability under natural 
conditions is ascertained. Then, intervention is introduced for one of the 
behaviors while all other behaviors continue to be measured at regular inter- 
vals. Once the success criterion is obtained for the first behavior, inter- 
vention is started for a second behavior; again, all other behaviors continue 
to be measured. This procedure is repeated for each of the remaining behav- 
iors". The baseline of the behaviors is the standard against which the effects 
of intervention are judged (Tawney & Gast, 1984) — see Figure l._ 

The multiple baseline prooedure assumes that target behaviors are func- 
tionally independent — a dif ficult as sumption to make in the area of infant 
development. For example, one cannot be sure that progress in head control is 
independent of progress in making sounds or progress in grasp patterns. How- 



Figuie 1 
Multiple Baseline Graph 



(hypothetical results) 
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ever, variations of the multiple baseline procedure .have been used in some 
infant programs to document progress due to intervention, (e.g., Barerra, 
Routh, Parr, Johnson, Are nd s hors t , Gobi sby, & Schroeder, 1976; Johnson-Martin 
et al., in press) and should be considered further. 

The multiple baseline procedure can be applied to an individual child by 
assessing his or her ability to perform various skills over several days or 
Weeks. Instruction is then started for one of these skills, while data is 
collected regularly on mastery of the other skills. After the one skill tar- 
getted for intervention is mastered, intervent ion is initiated for the next 
skill, and so forth. Charting progress over time should reveal changes in 
skill acquisition that coincide with intervention. Skills for which treatment 
is delayed serve as a baseline against which progress is measured in the 
skills targetted for intervention. 

In a second modification of the multiple baseline procedure , domains of 
development are divided into two or more groups (preferably through random 
assignment). A child or group of children is assessed in all domains at regu- 
lar intervals; -intervent ion is initiated in one group and continued for 
several months. Then, intervent ion is discontinued in these domains (although 
assessment continues) and initiated in the second set of domains . Over time, 
progress in the domains targetted for intervention may be compared with prog- 
ress made in domains not targetted for intervention. This procedure was used 
in the preliminary study by Barrerra et al. (1976) to document the interven- 
tion effectiveness in a small group of children; the field test procedures of 
the Carolina Curriculum for Handicapped Children (Johnson-Martin, et al, in 
press) also uses this procedure. In both instances, evidence was provided for 
statistically significant differences in progress made intervention areas. 

Many interventionists argue against delaying intervention in one area of 
development in order to use the delayed area as a baseline to assess another 
area. But early . childhood special educators must realise that no intervention 
program can achieve everything that needs to be done for a child at one time. 
Further, it is unnecessary to delay all intervention in, for example, the 
gross-motor area while working in the language area. Each of the traditional 
domains of development can be divided into smaller domains which can then be 
used as the basis of the multiple baseline procedure. 

In field testing the Carolina Curriculum, for- example* 24 skill areas 
were identified, and children were assessed in each of these areas. For three 
months, intervention" was carried out in all five motor areas and in half of 
the other areas, with an equal representation of strong and weak areas for 
each child. At the end of three months each child was reassessed in all 24. 
areas. Intervention for the next three-month period continued in the motor 
areas and in the areas where no intervention had been made in the first three 
months. Then, progress (i.e., the total number of items* passed) was compared 
between those nonmotor areas worked on and those not worked on during each 
three-month period (Johnson-Mart in et al., in press). 

Comparison to normative skill acquisition . This alternative to group 
design, suggested by Sommers, McGregor, Lesh, and Reed (1980), is a procedure 
that considers the cumulative total of skills and behaviors present (measured 
on any preferred assessment devise). This total is plotted on a graph against 



expected attainment for the normative sample over a longitudinal period. Pre- 
intervention assessment indicates the number of skills a handicapped child has 
mastered. These are plotted on a graph to show the comparison to normal chil- 
dren at a given time. After intervention is initiated, periodic assessments 
continue; this data is also plotted on the graph. A change in the slope of 
the curve presumably reflects changes in development due to intervention « 
Data plotted for a hypothetical case is presented in Figure 2. \. 



Figure 2 

Comparison of Intervention with Normal Development 
and Expected Development without intervention 

(hypothetical results) 



Normal development 




2 4 "6 8 10 ' 12 14 16 18 
Chronological age (months) 



Goal Attainment Scaling . A procedure developed by Kiresuk and Lund 
(1976), Goal Attainment Scaling involves setting goals for individuals in a 
number of domains and assessing the degree to which the goals are achieved. A 
formula permits the derivation of a Goal Attainment Score (GAS) for each 
individual. GASs may be compared regardless of the assessment instruments or 
other bases upon which the goals were developed. Likewise, a program can be 
evaluated by examining the average GAS of the clients. This procedure allows 
an analysis of group data without the constraint of having to use a common 
assessment instrument for every individual in the group. The Goal Attainment 
Scaling method has been widely used with adults in business and in human ser- 
vices. It has also, been used in a few s todies of children in mental health 
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and school settings (Simeonsson, Huntington, & a Short, 1982). According to 
Simeonsson et al. (1982), a number of features make Goal' Attainment Scaling a 
useful evaluation strategy for handicapped infants and children: it is direct- • 
ly relevant to therapeutic concerns and can use unique child measures; it can 
make use of the goals and objectives that are already a part, of most interven- 
tion programs; it allows for. dif ferent ial weighting of goals; and the values 
derived from the Hise of GASs are both numerically and . conceptually meaningful. 
The. most essential problem with Goal Attainment Scaling is determining the 
basis for selection of the goals for each child. Theoretically, intervention 
produces development or change beyond that expected from maturation. Such 
goal setting requires astute. judgments on the part of the interventionist. 
One program might look effective and another ineffective if the former program 
set unrealistically low goals and the latter set goals that were unreal is tical- 
ly high. To prevent this kind of bias, interventionists from other programs 
should be invited in to evaluate the children' independently and compare the 
goals that have been set and attained with those for similar children in their 
own programs or in other programs with which they are familiar* At this stage 
of our knowledge, the. consensus of experienced interventionists is probably the 
best measure that is available for judging' the validity of the goals selected * 
for the Goal Attainment Scaling procedure. As an additional safeguard for some 
types of clients, progress measured by Goal Attainment Scaling can be -compared 
with progress suggested by norm-referenced assessments. . 

Child Progress ox Program Effectiveness 

The developmental progress of children that presumably results from 1 
intervention is often the primary, if not the only* measure used to document 
program effectiveness. This is unfortunate since early intervention programs 
provide many services that may not : specif ically affect developmental progress, 
i.e., the mastery of developmental milestones. Programs that involve parents 
oftsn have a significant effect on the way these parents feel about themselves 
and their children, on their ability to serve as effective advocates for their 
children, and on their ability to interact effectively with their children. 

Changes in parental attitudes, coupled with child-directed program 
efforts, may create important behavioral changes in children — changes not 
reflected in developmental measures. For example, a non responsive child may 
become responsive, an irritable .child may become calmer, while a passive child 
may grow more active, and so forth. Surely, such changes represent child 
progress and should not be overlooked when documenting the effectiveness of a 
program. Developmental measures are only one factor, determining the long-term 
outcome of early intervention effectiveness (Lazar & Darlington, 1982). 
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Chapter 4 

Social Validity 

by Phillip Strain 



WHAT IS CHILD PROGRESS? 



Early childhood special education professionals who attend to the 
issue of child progress usually concentrate on the psychometric properties 
of child outcome measures (e.g., reliability, predictive validity). Un- 
fortunately, social validity or social significance of outcome indices Is 
too often ignored. 

E^mlning the social validity of an outcome addresses this basic ques 
tion: Does the outcome directly or indirectly help the child and family 
meet society's demands (Wolf * 1978)? This broad question of social valid- 
ity can be further divided: 

• Following Intervention, do the child and family func- 
tion more like their peers in the social community? 

• Do members of the child and family f s peer group per- 
ceive that the "intervention effects are positive, and 
do they behave more positively toward the clients 
after intervention? * ' 

Seldom are either of * these two questions addressed in early interven- 
tion outcome research* Yet , ignoring these social validity concerns may 
set the stage' for "wash-out" of Intervention effects, minimal public sup- 
port for intervention programs, and the perpetuation of statistically sig- 
nificant but socially trivial outcomes. 

Let us take, for example » two tried-and-true outcome indices in early 

intervention programs children's cognitive functioning and parents* 

teaching skills — and weigh them against the social Validity criterion. 

i - • . 

Social Validity Vs. Measurement of Cognitive Functioning 

Changes in children f s cognitive functioning, as assessed by standard- 
ized JQ instruments or developmental scales, are often used as an outcome 
in intervention programs for development ally delayed children. Is change 
in cognitive functioning a socially valid index of intervention Impact? 
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Regret ably, the answer is unclear. For handicapped populations within a 
relatively narrow band of performance (mild mental retardation to above 
average intelligence), IQ is a good predictor of socially accepted and 
valued performance, such as: timely progress through, school, academic 
achievement near or at grade-level, gainful. employment , and positive self- 
evaluations. For moderately and more severely handicapped groups, the ** 
relationship between IQ level and social adaptation is uncertain. IQ has 
little relevance to socially valid criteria such as less restrictive educa- 
tional environments and deinstitutionalization. -These opportunities tend 
to be based not upon IQ but upon the individual's social skills (lack of 
deviant, antisocial behaviors). Further, there is no evidence at this time 
to suggest that a certain level of IQ increase is even percept able to sig- 
nificant social agents. Most intervention efforts that use IQ as the out- 
come have shown gains of one standard deviation or less. It is doubtful 
that the expanded repertoire necessary to produce a 10-, 15-, or 20-point 
IQ increase will have any significant impact on the way the child functions 
in society. 

Social Validity Vs. Measurement of Parents' Teaching Skills 

- Parent's' teaching skill is a frequent target in early intervention 
programs. More often than not, some form of criterion-referenced assess- 
ment is used to assess teaching skills prior to instruction and after 
Instruction. A wealth of literature is now available that points to 
the efficiency of many training models and the impact of parents' newly 
acquired skills on their children's behavior.^ One might argue that social 
'ralidity is certain when positive changes in child behaviors can be linked 
directly to parents' improved teaching skill. But, there is scant evidence 
that teaching parents to teach has a positive, facilitat ive influence on 
the parent, the parent -child relationship, or the family. We do not know, 
for example, if parents who receive certain training are therefore more 
satisfied and fulfilled in their lives; if they now receive more support 
from family and friends; if they are more willing to persist with child 
instruction; if they have better marital relations; if they are more active 
in pursuing their life goals; if they perceive that the overall quality of 
their life has changed; and so on. Of course, it is possible also that 
negative outcomes, as yet unmeasured, may accrue from parent training. 

By no means is the goal here to criticize the use of IQ measures, par- 
ents' teaching skills, or any other outcome index. Rather, the intent is 
to draw attention to the. many socially significant outcomes of early inter- 
vention and to suggest that they, be considered in the area of assessment of 
child progress. Widespread use of a child outcome does not,* ergo, secure 
social validity. 

The examples below offer several suggestions for expanding our pro- 
fessional repertoire of socially valid indices ofj assessing' child and par- 
ent .outcomes of early intervention. 



Socially Valid Child Outcomes 



Most early intervention programs do an excellent job of improving 
children's accuracy on variouis tasks or developing behavioral competencies 
that did not exist prior to intervention. Few programs , however, have 
examined the fluency with whicji children perform their newly acquired 
skills. Yet, responding at a satisfactory pdce is a major determinant 
of the social acceptability or validity of many skills. An accurate but 
belabored signature may go unnoticed in the preschool or kindergarten, but 
it is certainly a skill deficit in , the early elementary grades (Kerr & 
Lambert, 1982). Similarly, accurate counting of "objects 1 to 10 is seldom 
made time dependent in our early intervention efforts, yet, fluency 'in num- 
erosity is absolutely essential for the most rudimentary of tnath operations 
(Strain & Kerr, 1981). Finally, many programs measure the number of words 
that a child learns to say in a " word-cal ling M c ont ex t and spend a con- 
siderable amount of time teaching new words.": However, reading experts gen- 
erally agree that an acceptable rate of response along with good compre- 
hension are the hallmarks of successful readers. 

In addition to including measures of fluency, early intervention pro- 
grams can also improve the social validity of child outcomes by selecting 
levels of target behavior (e.gV, percent of time spent in positive social 
interaction, percent of time working. independent of adult supervision) that 
are associated with ^successful performance in future ^educational and social 
settings. -Using both tether-completed checklists/ questionnaires and dir^ 
ect observation of successful children, a number of 'investigators (e.g., 
^train, in press; Vincent et al., 1980; Walker & Fops, 1976) have pin- 
pointed levels of academic and social performance that characterize com- 
petent, children in preschool j kindergarten, and the early elementary 
grades. 

Socially Valid Parent Outcomes i 

To date, parent outcomes in early intervention programs have been lim- 
ited primarily to a nafrow range of behaviors that occur within the mother-, 
child dyad. Consistent with the educational emphasis of most early inter- 
vention programs, teaching behaviors exhibited by parents are the pre- 
dominant Lorget of treatment- to parents and focus of parent evaluation. 

Though these teaching skills may be important to the child's long-term 
progress (e.g>, Lovaas, 1981; Strain, Steele, Ellis, & Timm, 19*82)., many 
indirect parent outcomes of early 'inter-vent ion remain undetected. For - 
example, many parents with, handicapped children beb'ome quite insular; that 
is, they do not tend to go pti outings, have extended friendships, or gener-. 
ally share- their positive and negative experiences with others. In many 
ways, their behavior is structurally similar to- several primary character- - 
istics of clinical depression. Recent work by Wahler and Dumas (in press), 
documents repreated incidence of reported depression by parents of disabled 
children. Family insularity and depressive characteristics are both power- 
ful iridicators of present and future adjustment problems. Assessment and 
amelioration of these problems could greatly add to the social validity of 
early intervention outcomes. , 
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HOW CAN CHILD PROGRESS BE MEASURED? 



Indeed, there are many advantages to using standardized tests to docu- 
ment the efficacy of an early intervention program.' For example, results 
can be readily compared with other intervention programs using the same 
measures, the psychometric properties o£. the tests are known, and uniform 
procedures for administration and scoring reduce the chance of spurious 
findings. .On occasion, however, the very procedures for test administra- 
tion and scoring can bias outcomes against detecting a significant inter- 
vention effect. 

When we use most standardized tests we must accept 'two basic assump- 
tions: the children -or parents are sufficiently motivated — by the 
testing material and despite limited examiner feedback — to do their best; 
and the response mode(s), the content, and the way the* test items are 
presented do not unduly discriminate against the testee. For many handi- 
capped young children, these assumptions do not conform to our best infor- 
mation about IfKe fragile nature of human behavior and the reactivity of 
assessment. So, when we use standardized tests to measure progress of 

handicapped children, 'we get results based on possibly false assumptions. 

*' * 

Motivation 

• * r * 

4 Qyer a half-century of research has been conducted on the effects of 
motivation on test performance. The overwhelming amount of this research' 
indicates that children attempt more test items and produce significantly 
higher test scores when they are reinforced for "good test-taking behav- 
ior e " Some might argue that the logical approach is simply to reinforce 
preschool handicapped children during test-taking, but such an approach 
would render the intervention evaluation invalid. When standardized tests 
are used, it is essential that the test assumptions be met to the maximum 
extent possible (Maheady, Sainato, Maitland, 1983). 

A major issue for the early intervention researcher, then, is to iden- 
tify the extent to which the problems of nonmotivated assessment may affect 
any particular evaluation. For many groups of preschool handicapped chil- 
dren, the bias created by nonmotivated testing is likely to be substantial. 
For example, several categories of handicapped children (e.g., autistic, 
severely and profoundly mentally retarded) have as a major defining charac- 
teristic the inability to acquire new skills withdut carefully programmed 
use of motivational incent Ives .or reinforcement. Indeed, extensive re- 
search has been devoted to uncovering methods for identifying the idiosyn- 
cratic motivational, variables necessary for learning with severely disabled 
groups (e.g., Carr, 1977). , * 

The contrast between the heavy use of motivational incentives in an 
intervention program and the absence of motivational incent Ives * in the 
assessment situation suggests that children^ test performance would be b 
less than optimal. Where children receive extencive feedback for their 
daily performance in intervention programs,- nonmotivated assessment may be 
particularly punishing and biased. 
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Mode , Content , and Presentation 

Many standardized tests result in underes.timations~of children's 
performance becausie the response mode*, content , and presentation of the 
assessment items discriminate against handicapped preschoolers. The major 
ity of • standardized tests require the testee to respond with an expressive 
language response .or a pointing response.. For many preschool handicapped 
children, expressive language production is under very tenuous control. 
That is, their verbal behavior may vary greatly with particular instruc- 
tors, settings, and stimulus materials. For other preschool handicapped 
children, their verbal repertoire may be quite extensive and not situation- 
specific; yet, their articulation is such that examiners who are not inti- 
mately familiar with certain verbal omissions or substitutions may unreli- 
ably score many responses. 

Standardized assessment instruments presuppose that testees |kve some 
exposure or experience with, the part icular assessment items* As Duncan et 
al. (1981) suggest, the contents of many standardized tests (particularly 
those aimed at cognitive assessment) assume that children have been exposed 
to: traditional family living arrangements; household items; animalsj cer- 
tain toys; and so on. This as sumpt ion is likely false for children from 
disadvantaged backgrounds, children who reside in .other than traditional 
homes and children whose handicaps have limited their access to environ- 
mental events common to nonhandicapped children. 

Standardized tests are intended to be presented in ways that tfill 
ensure that children understand what response is required but will not 
prompt the correct answer. However, more often than not, the examiner 
gives the child a verbal command to engage in some behayior. This presen- 
tation format assumes that testees are generally compliant and that the 

command * (e.g. , "Show me ." "Point to ." "What is this?") is 

understood. We kn w, of course, that many preschool children are in treat- 
ment programs because of their oppositional behavior and general noncom- 
pliance to adults' requests. For many severely disabled children, verbal 
comprehension is extremely restricted. There are countless children, for 
example, who can make many correct discriminations when asked, "Show me 

«" If these same children are asked, "Point to , " they may not 

respond at all. It is a regrettable irony that the limited, carefully, 
routinized cues often needed to teach severely disabled children may limit 
performance on many standardized tests. 



The tendency when using standardized tests is to interpret nonrfesporid- 
ing and incorrect responding .as valid evidence of / skill deficiency. 
Among the general population of preschool handicapped children there are 
children who: 

• know many answers but are not motivated to respond; 

• cannot respond cot rectly because they simply have no relevant 
experience with the test item; 



Summary 
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* could likely score much higher on .a standardized test if the 
response mode of the test conformed to their learning 
characteristics;; 

* dp aot respond generally to any requests made by adults 

* would likely gcst many more items correct if. the stimulus items 
were presented in a familiar forroat. 

tor all of these children and many others, evaluat ion -outcomes based 
on standardized tests may^imit the opportunity to detect real intervexit ion 
effects. Measurement of specific intervention effects using techniques of 
single-subject designs currently offers the best alternative to standard- 
ized testing. a 



VJ HAT DESIGNS 03. PROCEDURES SHOU LD BE USED TO DETERMINE PROGRESS ? 

In recent years, there has been a rapid increase in the use of single- 
subject designs to evaluate the effects of early intervention programs. 
To understand the txu& advaiitages and limitations of these designs, it is 
important to deal initially with a few of the more onerous problems. 

First of all, single-subject designs (perhaps a more descriptive term 
might be, "subjects-as~their~6wn~control designs") that appear in the cur- 
rent literature often include multiple subjects in numbers equal to those 
in experimental or control groups in the early education efficacy litera- 
ture. Second, the use of these designs is not limited to the evaluation of 
behavioral ly oriented Intervention projects. Single-subject designs have 
been used to study the effects of health safety regulations, toothbrushing, 
anti-litter campaigns, racial awareness programs, "I"-message use, and 
countless school-related interventions. There are no single-subject design 
requirements that automatically exclude the evaluation of even the most 
strident nonbehavioral intervention* Finally, there is the issue of exter- 
nal validity, or the confidence one can have in generalizing the findings 
from one outcome study to members of the same or a similar subject group. 

It has been argued that singles-subject designs do not offer any evi- 
dence that suggests findings can be generalized. In truth, single-subject 
and group designs offer two very different solutions to the ^important 
external validity problem. Oroup desigxis rely on the random or matched 
assignment of subjects to experimental groups and the laws of probability 
to determine the likelihood that &n outcome could have been produced by 
chance. The lower the "chance occurrence" of the findings the more con- 
fidence one can have that similar subjects will be affected in a like man- 
ner if the intervention were repeated. Single-subject designs, on the 
other hand, attempt to sho^r (usually with a more limited number of sub- 
jects) that the intervention in f/icf: will yield similar effects. Chance is 
reduced to zero. In addition, individual single-subject designs are viewed 
as one in a series of replication needed to validate any intervention. 
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These replications involve the same intervention implemented by different 
change agents with different target subjects. 

There are a number of characteristics of single-subj ect designs that 
make them particularly attractive for early intervention outcome research. 

Continuous Behavior Monitoring 

Single-subject designs require that the intervention targets (e.g., 
improved preacademic performance, greater social participation, less tan- 
trums, no episodes of self-injury) be monitored on a continuous, day-to-day 
basis. Though a labor-intensive requirement, continuous monitoring yields 
vital information not available in group designs. Specifically, the timing 
and pattern of intervention. effects are preserved. A pre-post, group 
design method can show that an intervention produced a certain effect, but 
the continuous monitoring in a single-subject design can pinpoint when the 
effect took place , if it was gradual or abrupt, and if level at posttest 
was the optimum change produced. 



Precise Description of the Ind ependent Variable (i.e., Intervention ) 

Individual single-subject experiments are considered to be one of a 
series of replications. To facilitate replication, the independent vari- 
able must be defined and measured with such precision that the intervention 
can be reproduced* with high fidelity. ,This requirement is of great benefit 
to program developers, and it is the only way to ensure an exact replica- 
tion. Therefore,, in descriptions of independent variables in single- 
subject research, the following requirements should be met: instructional 
behaviors of teachers, parents, etc. ,. are clearly specified; these instruc- 
tional behaviors are measured systematically as to their appropriate appli- 
cations; independent observer reliability confirms the accuracy of the mea- 
surement of the instructional-behaviors; materials used in instruction are 
specified, and their use is monitored reliably; and the setting (i.e.; 
physical arrangements, ratio of adults to children or handicapped to non- 
handicapped, etc.) for instruction is specified and reliably monitored. 
When any of these requirements is missing, the relat ions hip between inter- 
vention and outcome is clouded, and scientific replication becomes a hit- 
or-miss proposition. It should be emphasized' that scientific replication 
means something quite different than institutional adoption of a treatment 
approach or service delivery model, though the later is often called repli- 
cation. Scientific replication, which Sidman (1960) argues to be the very 
essence of demonstrating that the presence of variable X produced Y out- 
comes, demands that treatments be so well documented that another indi- 
vidual could create the same instructional context and, it is hoped, pro- 
duce the same outcomes with similar children (Strain, 1981). 

By contrast, intervention variable descriptors in group designs often 
(but not always) are limited to nominal presentations such as, "the Ameli- 
orative Curriculum was used," "developmental milestones were employed," or 
"teacher reinforcement wasgiven." Obviously, an exact replication could 
not follow on the heels of such an inadequate descript ion. 
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Public Policy and Best Practice 
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One of the hallmarks of P.L. 94-142' and a basic tenent of best educa- 
tional practice is the individualization of assessment and treatment. 
Practitioners from all reflated persuasions seem to agree that the benefits 
gained by careful individualized programming are worth the added costs; 
indeed, an impressive data base supports this nption* Single-subject 
designs in which an individuals own -beginning performance is the primary' 
standard against which the intervention effect is compared is the closest 
we have come to eliminating the dysfunctional separation between good prac- 
tice and research of scientific integrity. 

J do not consider that single-subject designs have inherent disadvan- 
tages vis a vis assessing early intervention outcomes. I do believe, how- 
ever, that consumers and potential users should be aware of the following 
caveats. 

* ' ■ - " 

Professional Myopia 

A modest, albeit occasionally powerful, minority in the scientific 
community look upon single-subject designs as something less than scien- 
tific and their users as something worse than Satanic. The existence of 
this minority is not a reason to avoid single-subject designs, but it may 
be worthwhile to analyze the biases of your intended audience before a 
final decision is made about a treatment outcome design. 



Discontinuous Dependent Measures 

Some potential child outcomes are not appropriately measured by, or 
compatible with, continuous measurement. Included are measurements of IQ, 
achievement, and motor development that rely on testing procedures that arc* 
confounded by repeated use over a short period of time. 



Design Fluctuations 

Most procedural details of between-group research can.be specified 
from the outset. However, in single-suhject designs, the incoming data 
dictates decisions such as: length of a baseline or preintervent ion 
assessment phase; when to change experimental phases; and adding another 
intervention component. The necessary flexibility built into single- 
subject designs demands close, usually daily monitoring. 

: • * ** 

Interactions Between Client and In tervent ion 

Often it is important to know if an intervention is more or less 
effective as a function of such child characteristics as: level of intel- 
ligence, age, handicapping condition,* or* past treatment history. 
Currently, no acceptable single-subject designs can address intervent ion- 



62 

59 



client interactions. Factorial designs should clearly be chosen in such 
instances • 
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